Participant Presentations Please Sign Up: Name Email (Onyen is fine, or …) Are You ENRolled? Tentative Title (???? Is OK) When: Next Week, Early, Oct.,

Tags:

Post on 18-Dec-2015

215 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

Transcript

Participant PresentationsPlease Sign Upbull Namebull Email (Onyen is fine or hellip)bull Are You ENRolledbull Tentative Title ( Is OK)bull When

Next Week Early Oct Nov Late

PCA to find clustersPCA of Mass Flux Data

Statistical SmoothingIn 1 Dimension 2 Major Settings

bull Density EstimationldquoHistogramsrdquo

bull Nonparametric RegressionldquoScatterplot Smoothingrdquo

Kernel Density EstimationChondrite Databull Sum pieces to estimate densitybull Suggests 3 modes (rock sources)

Scatterplot SmoothingEg Bralower Fossil Data ndash some

smooths

Statistical SmoothingFundamental Question

For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

Smoothingrdquo

Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

SiZer BackgroundFun Scale Space Views (Incomes Data)

Surface View

SiZer BackgroundSiZer analysis of British Incomes data

>

SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

statistically significant

Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

wiggles not significant

Again all are correct

just different scales

SiZer BackgroundScale Space and Kernel Choice

ie Shape of Window

Compelling Answer Gaussian

Only ldquoVariation Diminishingrdquo Kernel Shape

I e Modes decreases with bandwidth h

Lindebergh (1994)

Chaudhuri amp Marron (2000)

SiZer BackgroundRecall

Hidalgo

Stamps

Data

>

SiZer Overview

Would you like to try smoothing amp

SiZer

bull Marron Software Website as Before

bull In ldquoSmoothingrdquo Directory

ndash kdeSMm

ndash nprSMm

ndash sizerSMm

bull Recall ldquogtgt help sizerSMrdquo for

usage

PCA to find clustersReturn to PCA of Mass Flux Data

PCA to find clustersSiZer analysis of Mass Flux PC1

PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

PCA to find clustersSiZer analysis of Mass Flux PC1

Also in Curvature

PCA to find clustersSiZer analysis of Mass Flux PC1

And in Other Comprsquos

PCA to find clustersSiZer analysis of Mass Flux PC1

Conclusion

bull Found 3 significant clusters

bull Correspond to 3 known ldquocloud typesrdquo

bull Worth deeper investigation

Recall Yeast Cell Cycle Data

bull ldquoGene Expressionrdquo ndash Micro-array data

bull Data (after major preprocessing) Expression ldquolevelrdquo of

bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

10s)bull Interesting statistical issue

High Dimension Low Sample Size data

(HDLSS)

Yeast Cell Cycle Data FDA View

Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

Yeast Cell Cycle Data FDA View

Periodic genes

Naiumlve

approach

Simple PCA

Yeast Cell Cycle Data FDA Viewbull Central question which genes are

ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

ldquovariationrdquobull PCA finds ldquodirections of maximal

variationrdquobull Often but not always same as

ldquointeresting directionsrdquobull Here need better approach to study

periodicities

Yeast Cell Cycles Freq 2 Proj

PCA on

Freq 2

Periodic

Component

Of Data

Frequency 2 Analysisbull Project data onto 2-dim space of sin and

cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

Approach from Zhao Marron amp Wells (2004)

Frequency 2 Analysis

Frequency 2 Analysisbull Project data onto 2-dim space of sin and

cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

notable differencesbull Now try to improve ldquophase classificationrdquo

Yeast Cell CycleRevisit ldquophase classificationrdquo

approachbull Use outer 200 genes

(other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

(finds significant bumps etc in histogram)

bull Carefully redrew boundariesbull Check by studying kde angles

SiZer Study of Distrsquon of Angles

Reclassification of Major Genes

Compare to Previous Classifrsquon

New Subpopulation View

New Subpopulation View

NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

Detailed Look at PCA

Now Study ldquoFolklorerdquo More Carefully

bull BackGround

bull History

bull Underpinnings

(Mathematical amp Computational)

Good Overall Reference Jolliffe (2002)

PCA Rediscovery ndash Renaming

Statistics Principal Component Analysis (PCA)

PCA Rediscovery ndash Renaming

Statistics Principal Component Analysis (PCA)

Social Sciences Factor Analysis (PCA is a subset)

PCA Rediscovery ndash Renaming

Statistics Principal Component Analysis (PCA)

Social Sciences Factor Analysis (PCA is a subset)

Probability Electrical EngKarhunen ndash Loeve expansion

PCA Rediscovery ndash Renaming

Statistics Principal Component Analysis (PCA)

Social Sciences Factor Analysis (PCA is a subset)

Probability Electrical EngKarhunen ndash Loeve expansion

Applied MathematicsProper Orthogonal Decomposition (POD)

PCA Rediscovery ndash Renaming

Statistics Principal Component Analysis (PCA)

Social Sciences Factor Analysis (PCA is a subset)

Probability Electrical EngKarhunen ndash Loeve expansion

Applied MathematicsProper Orthogonal Decomposition (POD)

Geo-Sciences Empirical Orthogonal Functions (EOF)

An Interesting Historical Note

The 1st () application of PCA to Functional

Data Analysis

An Interesting Historical Note

The 1st () application of PCA to Functional

Data Analysis

Rao (1958)

1st Paper with ldquoCurves as Data Objectsrdquo

viewpoint

Detailed Look at PCA

Three Important (amp Interesting) Viewpoints

1 Mathematics

2 Numerics

3 Statistics

Goal Study Interrelationships

Detailed Look at PCA

Three Important (amp Interesting) Viewpoints

1 Mathematics

2 Numerics

3 Statistics

1st Review Linear Alg and Multivar Prob

Review of Linear Algebra

Vector Space

bull set of ldquovectorsrdquo

bull and ldquoscalarsrdquo (coefficients)

xa

Review of Linear Algebra

Vector Space

bull set of ldquovectorsrdquo

bull and ldquoscalarsrdquo (coefficients)

bull ldquoclosedrdquo under ldquolinear combinationrdquo

( in space)

xa

i

ii xa

Review of Linear Algebra

Vector Space

bull set of ldquovectorsrdquo

bull and ldquoscalarsrdquo (coefficients)

bull ldquoclosedrdquo under ldquolinear combinationrdquo

( in space)

eg

ldquo dim Euclidrsquon spacerdquo

xa

i

ii xa

d

d

d xx

x

x

x 1

1

d

Review of Linear Algebra (Cont)

Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

Review of Linear Algebra (Cont)

Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

Review of Linear Algebra (Cont)

Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

Note Planes not Through the Origin

are not Subspaces

(Do not Contain )00 x

Review of Linear Algebra (Cont)

Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

(all Linear Combos of them =

= Containing Hyperplane

through Origin)

Review of Linear Algebra (Cont)

Basis of Subspace Set of Vectors that

bull Span ie Everything is a Lin Com of them

bull are Linearly Indeprsquot ie Lin Com is Unique

Review of Linear Algebra (Cont)

Basis of Subspace Set of Vectors that

bull Span ie Everything is a Lin Com of them

bull are Linearly Indeprsquot ie Lin Com is Unique

bull eg ldquoUnit Vector Basisrdquo

bull Since

d

1

0

0

0

1

0

0

0

1

1

0

0

0

1

0

0

0

1

212

1

d

d

xxx

x

x

x

Review of Linear Algebra (Cont)

Basis Matrix of subspace of

Given a basis

create matrix of columns

dnvv 1

nddnd

n

n

vv

vv

vvB

1

111

1

Review of Linear Algebra (Cont)

Then linear combo is a matrix multiplicatrsquon

where

n

iii aBva

1

na

a

a 1

Review of Linear Algebra (Cont)

Then linear combo is a matrix multiplicatrsquon

where

Note Right Multiplication Gives

Linear Combination of Column Vectors

n

iii aBva

1

na

a

a 1

Review of Linear Algebra (Cont)

Then linear combo is a matrix multiplicatrsquon

where

Check sizes

n

iii aBva

1

na

a

a 1

)1()(1 nndd

Review of Linear Algebra (Cont)

Aside on Matrix Multiplication (linear transformatrsquon)

For matrices

Define the Matrix Product

mkk

m

aa

aa

A

1

111

nmm

n

bb

bb

B

1

111

m

iniik

m

iiik

m

inii

m

iii

baba

baba

AB

1

11

11

111

Review of Linear Algebra (Cont)

Aside on Matrix Multiplication (linear transformatrsquon)

For matrices

Define the Matrix Product

(Inner Products of Rows With Columns )

mkk

m

aa

aa

A

1

111

nmm

n

bb

bb

B

1

111

m

iniik

m

iiik

m

inii

m

iii

baba

baba

AB

1

11

11

111

A B

Review of Linear Algebra (Cont)

Aside on Matrix Multiplication (linear transformatrsquon)

For matrices

Define the Matrix Product

(Inner Products of Rows With Columns )

(Composition of Linear Transformations)

mkk

m

aa

aa

A

1

111

nmm

n

bb

bb

B

1

111

m

iniik

m

iiik

m

inii

m

iii

baba

baba

AB

1

11

11

111

A B

Review of Linear Algebra (Cont)

Aside on Matrix Multiplication (linear transformatrsquon)

For matrices

Define the Matrix Product

(Inner Products of Rows With Columns )

(Composition of Linear Transformations)

Often Useful to Check Sizes

mkk

m

aa

aa

A

1

111

nmm

n

bb

bb

B

1

111

m

iniik

m

iiik

m

inii

m

iii

baba

baba

AB

1

11

11

111

nmmknk

A B

Review of Linear Algebra (Cont)

Aside on Matrix Multiplication (linear transformatrsquon)

For matrices

Define the Matrix Product

(Inner Products of Rows With Columns )

(Composition of Linear Transformations)

Often Useful to Check Sizes

mkk

m

aa

aa

A

1

111

nmm

n

bb

bb

B

1

111

m

iniik

m

iiik

m

inii

m

iii

baba

baba

AB

1

11

11

111

nmmknk

A B

Review of Linear Algebra (Cont)

Matrix Trace

bull For a Square Matrix

bull Define

mmm

m

aa

aa

A

1

111

m

iiiaAtr

1)(

Review of Linear Algebra (Cont)

Matrix Trace

bull For a Square Matrix

bull Define

bull Trace Commutes with Matrix Multiplication

mmm

m

aa

aa

A

1

111

m

iiiaAtr

1)(

BAtrABtr

Review of Linear Algebra (Cont)

Dimension of Subspace (a Notion of ldquoSizerdquo)

bull Number of Elements in a Basis (Unique)

Review of Linear Algebra (Cont)

Dimension of Subspace (a Notion of ldquoSizerdquo)

bull Number of Elements in a Basis (Unique)

bull (Use Basis Above)

bull eg dim of a line is 1

bull eg dim of a plane is 2

dd dim

Review of Linear Algebra (Cont)

Dimension of Subspace (a Notion of ldquoSizerdquo)

bull Number of Elements in a Basis (Unique)

bull (Use Basis Above)

bull eg dim of a line is 1

bull eg dim of a plane is 2

bull Dimension is ldquoDegrees of Freedomrdquo

(in Statistical Uses eg ANOVA)

dd dim

Review of Linear Algebra (Cont)

Norm of a Vector

bull in d 21

21

1

2 xxxx td

jj

Review of Linear Algebra (Cont)

Norm of a Vector

bull in

bull Idea length of the vector

d 2121

1

2 xxxx td

jj

Review of Linear Algebra (Cont)

Norm of a Vector

bull in

bull Idea length of the vector

bull Note strange properties for high

eg ldquolength of diagonal of unit cuberdquo =

d 2121

1

2 xxxx td

jj

d

d

Review of Linear Algebra (Cont)

Norm of a Vector (cont)

bull Length Normalized Vector

(has Length 1 thus on Surf of Unit Sphere

amp is a Direction Vector)

x

x

Review of Linear Algebra (Cont)

Norm of a Vector (cont)

bull Length Normalized Vector

(has Length 1 thus on Surf of Unit Sphere

amp is a Direction Vector)

bull Define Distance as

x

x

yxyxyxyxd t

Review of Linear Algebra (Cont)

Inner (Dot Scalar) Product

bull for Vectors and

yxyxyx td

jjj

1

x y

Review of Linear Algebra (Cont)

Inner (Dot Scalar) Product

bull for Vectors and

bull Related to Norm via

yxyxyx td

jjj

1

x y

21 xxx

Review of Linear Algebra (Cont)

Inner (Dot Scalar) Product (cont)

bull measures ldquoangle between and rdquo as

yyxx

yx

yx

yxyxangle

tt

t

11 cos

cos

x y

Review of Linear Algebra (Cont)

Inner (Dot Scalar) Product (cont)

bull measures ldquoangle between and rdquo as

bull key to Orthogonality ie Perpendiculrsquoty

if and only if

yyxx

yx

yx

yxyxangle

tt

t

11 cos

cos

x y

yx 0 yx

Review of Linear Algebra (Cont)

Orthonormal Basis

bull All Orthogonal to each other

ie for

bull All have Length 1

ie for

nvv 1

1 ii vv

0 ii vv ii

ni 1

Review of Linear Algebra (Cont)

Orthonormal Basis (cont)

bull Spectral Representation

where

nvv 1

n

iii vax

1

ii vxa

Review of Linear Algebra (Cont)

Orthonormal Basis (cont)

bull Spectral Representation

where

(Coefficient is Inner Product Cool Notation)

nvv 1

n

iii vax

1

ii vxa

Review of Linear Algebra (Cont)

Orthonormal Basis (cont)

bull Spectral Representation

where

Check

nvv 1

n

iii vax

1

ii vxa

iii

n

iii

n

iiii avvavvavx

1

1

Review of Linear Algebra (Cont)

Orthonormal Basis (cont)

bull Spectral Representation

where

Check

bull Matrix Notation where ie

For the Basis Matrix

nvv 1

n

iii vax

1

ii vxa

iii

n

iii

n

iiii avvavvavx

1

1

aBx Bxa tt xBa t

nvvB 1

Review of Linear Algebra (Cont)

Orthonormal Basis (cont)

bull Spectral Representation

where

Check

bull Matrix Notation where ie

is called transform of

(eg Fourier or Wavelet)

nvv 1

n

iii vax

1

ii vxa

iii

n

iii

n

iiii avvavvavx

1

1

aBx Bxa tt xBa t

xa

Review of Linear Algebra (Cont)

Parseval identity for

in subsp genrsquod by o n basis

x

nvv 1

2

1

22

1

2 aavxx

n

ii

n

ii

Review of Linear Algebra (Cont)

Parseval identity for

in subsp genrsquod by o n basis

bull Pythagorean theorem

bull ldquoDecomposition of Energyrdquo

bull ANOVA - sums of squares

x

nvv 1

2

1

22

1

2 aavxx

n

ii

n

ii

Review of Linear Algebra (Cont)

Parseval identity for

in subsp genrsquod by o n basis

bull Pythagorean theorem

bull ldquoDecomposition of Energyrdquo

bull ANOVA - sums of squares

bull Transform has same length as

ie ldquorotation in rdquo

x

nvv 1

2

1

22

1

2 aavxx

n

ii

n

ii

a xd

Projection of a Vector onto a Subspace

bull Idea Member of that is Closest to

(ie ldquoBest Approxrsquonrdquo)

Review of Linear Algebra (Cont)x

xV

V

Projection of a Vector onto a Subspace

bull Idea Member of that is Closest to

(ie ldquoBest Approxrsquonrdquo)

bull Find that Solves

(ldquoLeast Squaresrdquo)

Review of Linear Algebra (Cont)x

xV

V

VxPV vxVv

min

Projection of a Vector onto a Subspace

bull Idea Member of that is Closest to

(ie ldquoBest Approxrsquonrdquo)

bull Find that Solves

(ldquoLeast Squaresrdquo)

bull For Inner Product (Hilbert) Space

Exists and is Unique

Review of Linear Algebra (Cont)x

xV

V

VxPV vxVv

min

xPV

Projection of a Vector onto a Subspace (cont)

bull General Solution in for Basis Matrix

Review of Linear Algebra (Cont)

d VB

xBBBBxP tVV

tVVV

1

Projection of a Vector onto a Subspace (cont)

bull General Solution in for Basis Matrix

bull So Projrsquon Operator is Matrix Multrsquon

(thus projection is another linear operation)

Review of Linear Algebra (Cont)

d VB

xBBBBxP tVV

tVVV

1

tVV

tVVV BBBBP

1

Projection of a Vector onto a Subspace (cont)

bull General Solution in for Basis Matrix

bull So Projrsquon Operator is Matrix Multrsquon

(thus projection is another linear operation)

(note same operation underlies least squares)

Review of Linear Algebra (Cont)

d VB

xBBBBxP tVV

tVVV

1

tVV

tVVV BBBBP

1

Review of Linear Algebra (Cont)

Projection using Orthonormal Basis

bull Basis Matrix is Orthonormal

nnVtV IBB

10

01

1

111

1

1

nnn

n

ntn

t

vvvv

vvvv

vv

v

v

nvv 1

Review of Linear Algebra (Cont)

Projection using Orthonormal Basis

bull Basis Matrix is Orthonormal

bull So =

= Recon(Coeffs of ldquoin Dirrsquonrdquo)

(Recall Right Multrsquon)

nnVtV IBB

10

01

1

111

1

1

nnn

n

ntn

t

vvvv

vvvv

vv

v

v

xBBxP tVVV

x V

nvv 1

Review of Linear Algebra (Cont)

Projection using Orthonormal Basis (cont)

bull For Orthogonal Complement

and

V

xPxPx VV 222xPxPx VV

Review of Linear Algebra (Cont)

Projection using Orthonormal Basis (cont)

bull For Orthogonal Complement

and

bull Parseval Inequality

V

xPxPx VV 222xPxPx VV

2

1

22

1

22 aavxxxP

n

ii

n

iiV

Review of Linear Algebra (Cont)

(Real) Unitary Matrices with

bull Orthonormal Basis Matrix

(So All of Above Applies)

ddU IUU t

Review of Linear Algebra (Cont)

(Real) Unitary Matrices with

bull Orthonormal Basis Matrix

(So All of Above Applies)

bull Note Transformrsquon is Distance Preserving

ddU IUU t

yxdyxyxyxUyUxUdn

i ii 2

1

Review of Linear Algebra (Cont)

(Real) Unitary Matrices with

bull Orthonormal Basis Matrix

(So All of Above Applies)

bull Note Transformrsquon is Distance Preserving

bull Lin Trans (Mult by ) is ~ Rotation

bull But also Includes ldquoMirror Imagesrdquo

ddU IUU t

yxdyxyxyxUyUxUdn

i ii 2

1

U

Review of Linear Algebra (Cont)

Singular Value Decomposition (SVD)

For a Matrix

Find

ndX

Review of Linear Algebra (Cont)

Singular Value Decomposition (SVD)

For a Matrix

Find a Diagonal Matrix

with Entries

called Singular Values

ndX

ndS

)min(1 ndss

Review of Linear Algebra (Cont)

Singular Value Decomposition (SVD)

For a Matrix

Find a Diagonal Matrix

with Entries

called Singular Values

And Unitary (Rotation) Matrices

(recall )

ndX

ndS

)min(1 ndss

ddU nnV

IVVUU tt

Review of Linear Algebra (Cont)

Singular Value Decomposition (SVD)

For a Matrix

Find a Diagonal Matrix

with Entries

called Singular Values

And Unitary (Rotation) Matrices

(recall )

So That

ndX

ndS

)min(1 ndss

ddU nnV

IVVUU tt tUSVX

Review of Linear Algebra (Cont)

Intuition behind Singular Value Decomposition

bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

vVSUvVSUvX tt

Review of Linear Algebra (Cont)

Intuition behind Singular Value Decomposition

bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

bull First rotate

X

vVSUvVSUvX tt

Review of Linear Algebra (Cont)

Intuition behind Singular Value Decomposition

bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

bull First rotate

bull Second rescale coordinate axes (by )

X

vVSUvVSUvX tt

is

Review of Linear Algebra (Cont)

Intuition behind Singular Value Decomposition

bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

bull First rotate

bull Second rescale coordinate axes (by )

bull Third rotate again

X

vVSUvVSUvX tt

is

Review of Linear Algebra (Cont)

Intuition behind Singular Value Decomposition

bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

bull First rotate

bull Second rescale coordinate axes (by )

bull Third rotate again

bull ie have diagonalized the transformation

X

vVSUvVSUvX tt

is

SVD Compact Representation

Useful Labeling

Singular Values in Increasing Order

Review of Linear Algebra (Cont)

)min(1 dnss

SVD Compact Representation

Useful Labeling

Singular Values in Increasing Order

Note singular values = 0 can be omitted

(Since do ldquo0-Stretchingrdquo)

Review of Linear Algebra (Cont)

)min(1 dnss

Review of Linear Algebra (Cont)

r

SVD Compact Representation

Useful Labeling

Singular Values in Increasing Order

Note singular values = 0 can be omitted

Let = of positive singular values

)min(1 dnss

Review of Linear Algebra (Cont)

r

SVD Compact Representation

Useful Labeling

Singular Values in Increasing Order

Note singular values = 0 can be omitted

Let = of positive singular values

Then

Where are truncations of

trnrrrd VSUX

VSU

)min(1 dnss

Review of Linear Algebra (Cont)

SVD Full Representation

=

Graphics Display Assumes

ndX ddU ndS nn

tV

nd

Review of Linear Algebra (Cont)

SVD Full Representation

=

Full Rank Basis Matrix

ndX ddU ndS nn

tV

Review of Linear Algebra (Cont)

SVD Full Representation

=

Full Rank Basis Matrix

All 0s in Bottom

ndX ddU ndS nn

tV

Review of Linear Algebra (Cont)

SVD Reduced Representation

=

These Columns Get 0ed Out

ndX ddU nnS nn

tV

nnd 0

Review of Linear Algebra (Cont)

SVD Reduced Representation

=ndX ndU nnS nn

tV

Review of Linear Algebra (Cont)

SVD Reduced Representation

=

Also Some of These May be 0

ndX ndU nnS nn

tV

Review of Linear Algebra (Cont)

SVD Compact Representation

= ndX rdU

rrS nrtV

0

Review of Linear Algebra (Cont)

SVD Compact Representation

=

These Get 0ed Out

ndX rdU

rrS nrtV

0

Review of Linear Algebra (Cont)

SVD Compact Representation

= ndX rdU

rrS nrtV

Review of Linear Algebra (Cont)

Eigenvalue Decomposition

For a (Symmetric) Square Matrix

Find

ddX

Review of Linear Algebra (Cont)

Eigenvalue Decomposition

For a (Symmetric) Square Matrix

Find a Diagonal Matrix

Called Eigenvalues

Convenient Ordering

ddX

d

D

0

01

n 1

Review of Linear Algebra (Cont)

Eigenvalue Decomposition

For a (Symmetric) Square Matrix

Find a Diagonal Matrix

And an Orthonormal Matrix

(ie )

ddX

d

D

0

01

ddB

ddtt IBBBB

Review of Linear Algebra (Cont)

Eigenvalue Decomposition

For a (Symmetric) Square Matrix

Find a Diagonal Matrix

And an Orthonormal Matrix

(ie )

So that ie

ddX

d

D

0

01

ddB

ddtt IBBBB

DBBX tBDBX

Review of Linear Algebra (Cont)

Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

(looks similar)

Review of Linear Algebra (Cont)

Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

(looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

Review of Linear Algebra (Cont)

Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

(looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

Complex (uses )

VU

1i

Review of Linear Algebra (Cont)

Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

(looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

VU

X

BVU

1i

  • Participant Presentations
  • PCA to find clusters
  • Statistical Smoothing
  • Kernel Density Estimation
  • Scatterplot Smoothing
  • Statistical Smoothing (2)
  • SiZer Background
  • SiZer Background (2)
  • SiZer Background (3)
  • SiZer Background (4)
  • SiZer Background (5)
  • SiZer Overview
  • PCA to find clusters (2)
  • PCA to find clusters (3)
  • PCA to find clusters (4)
  • PCA to find clusters (5)
  • PCA to find clusters (6)
  • PCA to find clusters (7)
  • Recall Yeast Cell Cycle Data
  • Yeast Cell Cycle Data FDA View
  • Yeast Cell Cycle Data FDA View (2)
  • Yeast Cell Cycle Data FDA View (3)
  • Yeast Cell Cycles Freq 2 Proj
  • Frequency 2 Analysis
  • Frequency 2 Analysis (2)
  • Frequency 2 Analysis (3)
  • Yeast Cell Cycle
  • SiZer Study of Distrsquon of Angles
  • Reclassification of Major Genes
  • Compare to Previous Classifrsquon
  • New Subpopulation View
  • New Subpopulation View (2)
  • Detailed Look at PCA
  • PCA Rediscovery ndash Renaming
  • PCA Rediscovery ndash Renaming (2)
  • PCA Rediscovery ndash Renaming (3)
  • PCA Rediscovery ndash Renaming (4)
  • PCA Rediscovery ndash Renaming (5)
  • An Interesting Historical Note
  • An Interesting Historical Note (2)
  • Detailed Look at PCA (2)
  • Detailed Look at PCA (3)
  • Review of Linear Algebra
  • Review of Linear Algebra (2)
  • Review of Linear Algebra (3)
  • Review of Linear Algebra (Cont)
  • Review of Linear Algebra (Cont) (2)
  • Review of Linear Algebra (Cont) (3)
  • Review of Linear Algebra (Cont) (4)
  • Review of Linear Algebra (Cont) (5)
  • Review of Linear Algebra (Cont) (6)
  • Review of Linear Algebra (Cont) (7)
  • Review of Linear Algebra (Cont) (8)
  • Review of Linear Algebra (Cont) (9)
  • Review of Linear Algebra (Cont) (10)
  • Review of Linear Algebra (Cont) (11)
  • Review of Linear Algebra (Cont) (12)
  • Review of Linear Algebra (Cont) (13)
  • Review of Linear Algebra (Cont) (14)
  • Review of Linear Algebra (Cont) (15)
  • Review of Linear Algebra (Cont) (16)
  • Review of Linear Algebra (Cont) (17)
  • Review of Linear Algebra (Cont) (18)
  • Review of Linear Algebra (Cont) (19)
  • Review of Linear Algebra (Cont) (20)
  • Review of Linear Algebra (Cont) (21)
  • Review of Linear Algebra (Cont) (22)
  • Review of Linear Algebra (Cont) (23)
  • Review of Linear Algebra (Cont) (24)
  • Review of Linear Algebra (Cont) (25)
  • Review of Linear Algebra (Cont) (26)
  • Review of Linear Algebra (Cont) (27)
  • Review of Linear Algebra (Cont) (28)
  • Review of Linear Algebra (Cont) (29)
  • Review of Linear Algebra (Cont) (30)
  • Review of Linear Algebra (Cont) (31)
  • Review of Linear Algebra (Cont) (32)
  • Review of Linear Algebra (Cont) (33)
  • Review of Linear Algebra (Cont) (34)
  • Review of Linear Algebra (Cont) (35)
  • Review of Linear Algebra (Cont) (36)
  • Review of Linear Algebra (Cont) (37)
  • Review of Linear Algebra (Cont) (38)
  • Review of Linear Algebra (Cont) (39)
  • Review of Linear Algebra (Cont) (40)
  • Review of Linear Algebra (Cont) (41)
  • Review of Linear Algebra (Cont) (42)
  • Review of Linear Algebra (Cont) (43)
  • Review of Linear Algebra (Cont) (44)
  • Review of Linear Algebra (Cont) (45)
  • Review of Linear Algebra (Cont) (46)
  • Review of Linear Algebra (Cont) (47)
  • Review of Linear Algebra (Cont) (48)
  • Review of Linear Algebra (Cont) (49)
  • Review of Linear Algebra (Cont) (50)
  • Review of Linear Algebra (Cont) (51)
  • Review of Linear Algebra (Cont) (52)
  • Review of Linear Algebra (Cont) (53)
  • Review of Linear Algebra (Cont) (54)
  • Review of Linear Algebra (Cont) (55)
  • Review of Linear Algebra (Cont) (56)
  • Review of Linear Algebra (Cont) (57)
  • Review of Linear Algebra (Cont) (58)
  • Review of Linear Algebra (Cont) (59)
  • Review of Linear Algebra (Cont) (60)
  • Review of Linear Algebra (Cont) (61)
  • Review of Linear Algebra (Cont) (62)
  • Review of Linear Algebra (Cont) (63)
  • Review of Linear Algebra (Cont) (64)
  • Review of Linear Algebra (Cont) (65)
  • Review of Linear Algebra (Cont) (66)
  • Review of Linear Algebra (Cont) (67)
  • Review of Linear Algebra (Cont) (68)
  • Review of Linear Algebra (Cont) (69)
  • Review of Linear Algebra (Cont) (70)
  • Review of Linear Algebra (Cont) (71)
  • Review of Linear Algebra (Cont) (72)
  • Review of Linear Algebra (Cont) (73)
  • Review of Linear Algebra (Cont) (74)
  • Review of Linear Algebra (Cont) (75)
  • Review of Linear Algebra (Cont) (76)
  • Review of Linear Algebra (Cont) (77)
  • Review of Linear Algebra (Cont) (78)
  • Review of Linear Algebra (Cont) (79)
  • Review of Linear Algebra (Cont) (80)
  • Review of Linear Algebra (Cont) (81)

    PCA to find clustersPCA of Mass Flux Data

    Statistical SmoothingIn 1 Dimension 2 Major Settings

    bull Density EstimationldquoHistogramsrdquo

    bull Nonparametric RegressionldquoScatterplot Smoothingrdquo

    Kernel Density EstimationChondrite Databull Sum pieces to estimate densitybull Suggests 3 modes (rock sources)

    Scatterplot SmoothingEg Bralower Fossil Data ndash some

    smooths

    Statistical SmoothingFundamental Question

    For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

    Smoothingrdquo

    Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

    SiZer BackgroundFun Scale Space Views (Incomes Data)

    Surface View

    SiZer BackgroundSiZer analysis of British Incomes data

    >

    SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

    statistically significant

    Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

    wiggles not significant

    Again all are correct

    just different scales

    SiZer BackgroundScale Space and Kernel Choice

    ie Shape of Window

    Compelling Answer Gaussian

    Only ldquoVariation Diminishingrdquo Kernel Shape

    I e Modes decreases with bandwidth h

    Lindebergh (1994)

    Chaudhuri amp Marron (2000)

    SiZer BackgroundRecall

    Hidalgo

    Stamps

    Data

    >

    SiZer Overview

    Would you like to try smoothing amp

    SiZer

    bull Marron Software Website as Before

    bull In ldquoSmoothingrdquo Directory

    ndash kdeSMm

    ndash nprSMm

    ndash sizerSMm

    bull Recall ldquogtgt help sizerSMrdquo for

    usage

    PCA to find clustersReturn to PCA of Mass Flux Data

    PCA to find clustersSiZer analysis of Mass Flux PC1

    PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

    PCA to find clustersSiZer analysis of Mass Flux PC1

    Also in Curvature

    PCA to find clustersSiZer analysis of Mass Flux PC1

    And in Other Comprsquos

    PCA to find clustersSiZer analysis of Mass Flux PC1

    Conclusion

    bull Found 3 significant clusters

    bull Correspond to 3 known ldquocloud typesrdquo

    bull Worth deeper investigation

    Recall Yeast Cell Cycle Data

    bull ldquoGene Expressionrdquo ndash Micro-array data

    bull Data (after major preprocessing) Expression ldquolevelrdquo of

    bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

    10s)bull Interesting statistical issue

    High Dimension Low Sample Size data

    (HDLSS)

    Yeast Cell Cycle Data FDA View

    Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

    Yeast Cell Cycle Data FDA View

    Periodic genes

    Naiumlve

    approach

    Simple PCA

    Yeast Cell Cycle Data FDA Viewbull Central question which genes are

    ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

    ldquovariationrdquobull PCA finds ldquodirections of maximal

    variationrdquobull Often but not always same as

    ldquointeresting directionsrdquobull Here need better approach to study

    periodicities

    Yeast Cell Cycles Freq 2 Proj

    PCA on

    Freq 2

    Periodic

    Component

    Of Data

    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

    Approach from Zhao Marron amp Wells (2004)

    Frequency 2 Analysis

    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

    classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

    notable differencesbull Now try to improve ldquophase classificationrdquo

    Yeast Cell CycleRevisit ldquophase classificationrdquo

    approachbull Use outer 200 genes

    (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

    (finds significant bumps etc in histogram)

    bull Carefully redrew boundariesbull Check by studying kde angles

    SiZer Study of Distrsquon of Angles

    Reclassification of Major Genes

    Compare to Previous Classifrsquon

    New Subpopulation View

    New Subpopulation View

    NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

    Detailed Look at PCA

    Now Study ldquoFolklorerdquo More Carefully

    bull BackGround

    bull History

    bull Underpinnings

    (Mathematical amp Computational)

    Good Overall Reference Jolliffe (2002)

    PCA Rediscovery ndash Renaming

    Statistics Principal Component Analysis (PCA)

    PCA Rediscovery ndash Renaming

    Statistics Principal Component Analysis (PCA)

    Social Sciences Factor Analysis (PCA is a subset)

    PCA Rediscovery ndash Renaming

    Statistics Principal Component Analysis (PCA)

    Social Sciences Factor Analysis (PCA is a subset)

    Probability Electrical EngKarhunen ndash Loeve expansion

    PCA Rediscovery ndash Renaming

    Statistics Principal Component Analysis (PCA)

    Social Sciences Factor Analysis (PCA is a subset)

    Probability Electrical EngKarhunen ndash Loeve expansion

    Applied MathematicsProper Orthogonal Decomposition (POD)

    PCA Rediscovery ndash Renaming

    Statistics Principal Component Analysis (PCA)

    Social Sciences Factor Analysis (PCA is a subset)

    Probability Electrical EngKarhunen ndash Loeve expansion

    Applied MathematicsProper Orthogonal Decomposition (POD)

    Geo-Sciences Empirical Orthogonal Functions (EOF)

    An Interesting Historical Note

    The 1st () application of PCA to Functional

    Data Analysis

    An Interesting Historical Note

    The 1st () application of PCA to Functional

    Data Analysis

    Rao (1958)

    1st Paper with ldquoCurves as Data Objectsrdquo

    viewpoint

    Detailed Look at PCA

    Three Important (amp Interesting) Viewpoints

    1 Mathematics

    2 Numerics

    3 Statistics

    Goal Study Interrelationships

    Detailed Look at PCA

    Three Important (amp Interesting) Viewpoints

    1 Mathematics

    2 Numerics

    3 Statistics

    1st Review Linear Alg and Multivar Prob

    Review of Linear Algebra

    Vector Space

    bull set of ldquovectorsrdquo

    bull and ldquoscalarsrdquo (coefficients)

    xa

    Review of Linear Algebra

    Vector Space

    bull set of ldquovectorsrdquo

    bull and ldquoscalarsrdquo (coefficients)

    bull ldquoclosedrdquo under ldquolinear combinationrdquo

    ( in space)

    xa

    i

    ii xa

    Review of Linear Algebra

    Vector Space

    bull set of ldquovectorsrdquo

    bull and ldquoscalarsrdquo (coefficients)

    bull ldquoclosedrdquo under ldquolinear combinationrdquo

    ( in space)

    eg

    ldquo dim Euclidrsquon spacerdquo

    xa

    i

    ii xa

    d

    d

    d xx

    x

    x

    x 1

    1

    d

    Review of Linear Algebra (Cont)

    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

    Review of Linear Algebra (Cont)

    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

    Review of Linear Algebra (Cont)

    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

    Note Planes not Through the Origin

    are not Subspaces

    (Do not Contain )00 x

    Review of Linear Algebra (Cont)

    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

    (all Linear Combos of them =

    = Containing Hyperplane

    through Origin)

    Review of Linear Algebra (Cont)

    Basis of Subspace Set of Vectors that

    bull Span ie Everything is a Lin Com of them

    bull are Linearly Indeprsquot ie Lin Com is Unique

    Review of Linear Algebra (Cont)

    Basis of Subspace Set of Vectors that

    bull Span ie Everything is a Lin Com of them

    bull are Linearly Indeprsquot ie Lin Com is Unique

    bull eg ldquoUnit Vector Basisrdquo

    bull Since

    d

    1

    0

    0

    0

    1

    0

    0

    0

    1

    1

    0

    0

    0

    1

    0

    0

    0

    1

    212

    1

    d

    d

    xxx

    x

    x

    x

    Review of Linear Algebra (Cont)

    Basis Matrix of subspace of

    Given a basis

    create matrix of columns

    dnvv 1

    nddnd

    n

    n

    vv

    vv

    vvB

    1

    111

    1

    Review of Linear Algebra (Cont)

    Then linear combo is a matrix multiplicatrsquon

    where

    n

    iii aBva

    1

    na

    a

    a 1

    Review of Linear Algebra (Cont)

    Then linear combo is a matrix multiplicatrsquon

    where

    Note Right Multiplication Gives

    Linear Combination of Column Vectors

    n

    iii aBva

    1

    na

    a

    a 1

    Review of Linear Algebra (Cont)

    Then linear combo is a matrix multiplicatrsquon

    where

    Check sizes

    n

    iii aBva

    1

    na

    a

    a 1

    )1()(1 nndd

    Review of Linear Algebra (Cont)

    Aside on Matrix Multiplication (linear transformatrsquon)

    For matrices

    Define the Matrix Product

    mkk

    m

    aa

    aa

    A

    1

    111

    nmm

    n

    bb

    bb

    B

    1

    111

    m

    iniik

    m

    iiik

    m

    inii

    m

    iii

    baba

    baba

    AB

    1

    11

    11

    111

    Review of Linear Algebra (Cont)

    Aside on Matrix Multiplication (linear transformatrsquon)

    For matrices

    Define the Matrix Product

    (Inner Products of Rows With Columns )

    mkk

    m

    aa

    aa

    A

    1

    111

    nmm

    n

    bb

    bb

    B

    1

    111

    m

    iniik

    m

    iiik

    m

    inii

    m

    iii

    baba

    baba

    AB

    1

    11

    11

    111

    A B

    Review of Linear Algebra (Cont)

    Aside on Matrix Multiplication (linear transformatrsquon)

    For matrices

    Define the Matrix Product

    (Inner Products of Rows With Columns )

    (Composition of Linear Transformations)

    mkk

    m

    aa

    aa

    A

    1

    111

    nmm

    n

    bb

    bb

    B

    1

    111

    m

    iniik

    m

    iiik

    m

    inii

    m

    iii

    baba

    baba

    AB

    1

    11

    11

    111

    A B

    Review of Linear Algebra (Cont)

    Aside on Matrix Multiplication (linear transformatrsquon)

    For matrices

    Define the Matrix Product

    (Inner Products of Rows With Columns )

    (Composition of Linear Transformations)

    Often Useful to Check Sizes

    mkk

    m

    aa

    aa

    A

    1

    111

    nmm

    n

    bb

    bb

    B

    1

    111

    m

    iniik

    m

    iiik

    m

    inii

    m

    iii

    baba

    baba

    AB

    1

    11

    11

    111

    nmmknk

    A B

    Review of Linear Algebra (Cont)

    Aside on Matrix Multiplication (linear transformatrsquon)

    For matrices

    Define the Matrix Product

    (Inner Products of Rows With Columns )

    (Composition of Linear Transformations)

    Often Useful to Check Sizes

    mkk

    m

    aa

    aa

    A

    1

    111

    nmm

    n

    bb

    bb

    B

    1

    111

    m

    iniik

    m

    iiik

    m

    inii

    m

    iii

    baba

    baba

    AB

    1

    11

    11

    111

    nmmknk

    A B

    Review of Linear Algebra (Cont)

    Matrix Trace

    bull For a Square Matrix

    bull Define

    mmm

    m

    aa

    aa

    A

    1

    111

    m

    iiiaAtr

    1)(

    Review of Linear Algebra (Cont)

    Matrix Trace

    bull For a Square Matrix

    bull Define

    bull Trace Commutes with Matrix Multiplication

    mmm

    m

    aa

    aa

    A

    1

    111

    m

    iiiaAtr

    1)(

    BAtrABtr

    Review of Linear Algebra (Cont)

    Dimension of Subspace (a Notion of ldquoSizerdquo)

    bull Number of Elements in a Basis (Unique)

    Review of Linear Algebra (Cont)

    Dimension of Subspace (a Notion of ldquoSizerdquo)

    bull Number of Elements in a Basis (Unique)

    bull (Use Basis Above)

    bull eg dim of a line is 1

    bull eg dim of a plane is 2

    dd dim

    Review of Linear Algebra (Cont)

    Dimension of Subspace (a Notion of ldquoSizerdquo)

    bull Number of Elements in a Basis (Unique)

    bull (Use Basis Above)

    bull eg dim of a line is 1

    bull eg dim of a plane is 2

    bull Dimension is ldquoDegrees of Freedomrdquo

    (in Statistical Uses eg ANOVA)

    dd dim

    Review of Linear Algebra (Cont)

    Norm of a Vector

    bull in d 21

    21

    1

    2 xxxx td

    jj

    Review of Linear Algebra (Cont)

    Norm of a Vector

    bull in

    bull Idea length of the vector

    d 2121

    1

    2 xxxx td

    jj

    Review of Linear Algebra (Cont)

    Norm of a Vector

    bull in

    bull Idea length of the vector

    bull Note strange properties for high

    eg ldquolength of diagonal of unit cuberdquo =

    d 2121

    1

    2 xxxx td

    jj

    d

    d

    Review of Linear Algebra (Cont)

    Norm of a Vector (cont)

    bull Length Normalized Vector

    (has Length 1 thus on Surf of Unit Sphere

    amp is a Direction Vector)

    x

    x

    Review of Linear Algebra (Cont)

    Norm of a Vector (cont)

    bull Length Normalized Vector

    (has Length 1 thus on Surf of Unit Sphere

    amp is a Direction Vector)

    bull Define Distance as

    x

    x

    yxyxyxyxd t

    Review of Linear Algebra (Cont)

    Inner (Dot Scalar) Product

    bull for Vectors and

    yxyxyx td

    jjj

    1

    x y

    Review of Linear Algebra (Cont)

    Inner (Dot Scalar) Product

    bull for Vectors and

    bull Related to Norm via

    yxyxyx td

    jjj

    1

    x y

    21 xxx

    Review of Linear Algebra (Cont)

    Inner (Dot Scalar) Product (cont)

    bull measures ldquoangle between and rdquo as

    yyxx

    yx

    yx

    yxyxangle

    tt

    t

    11 cos

    cos

    x y

    Review of Linear Algebra (Cont)

    Inner (Dot Scalar) Product (cont)

    bull measures ldquoangle between and rdquo as

    bull key to Orthogonality ie Perpendiculrsquoty

    if and only if

    yyxx

    yx

    yx

    yxyxangle

    tt

    t

    11 cos

    cos

    x y

    yx 0 yx

    Review of Linear Algebra (Cont)

    Orthonormal Basis

    bull All Orthogonal to each other

    ie for

    bull All have Length 1

    ie for

    nvv 1

    1 ii vv

    0 ii vv ii

    ni 1

    Review of Linear Algebra (Cont)

    Orthonormal Basis (cont)

    bull Spectral Representation

    where

    nvv 1

    n

    iii vax

    1

    ii vxa

    Review of Linear Algebra (Cont)

    Orthonormal Basis (cont)

    bull Spectral Representation

    where

    (Coefficient is Inner Product Cool Notation)

    nvv 1

    n

    iii vax

    1

    ii vxa

    Review of Linear Algebra (Cont)

    Orthonormal Basis (cont)

    bull Spectral Representation

    where

    Check

    nvv 1

    n

    iii vax

    1

    ii vxa

    iii

    n

    iii

    n

    iiii avvavvavx

    1

    1

    Review of Linear Algebra (Cont)

    Orthonormal Basis (cont)

    bull Spectral Representation

    where

    Check

    bull Matrix Notation where ie

    For the Basis Matrix

    nvv 1

    n

    iii vax

    1

    ii vxa

    iii

    n

    iii

    n

    iiii avvavvavx

    1

    1

    aBx Bxa tt xBa t

    nvvB 1

    Review of Linear Algebra (Cont)

    Orthonormal Basis (cont)

    bull Spectral Representation

    where

    Check

    bull Matrix Notation where ie

    is called transform of

    (eg Fourier or Wavelet)

    nvv 1

    n

    iii vax

    1

    ii vxa

    iii

    n

    iii

    n

    iiii avvavvavx

    1

    1

    aBx Bxa tt xBa t

    xa

    Review of Linear Algebra (Cont)

    Parseval identity for

    in subsp genrsquod by o n basis

    x

    nvv 1

    2

    1

    22

    1

    2 aavxx

    n

    ii

    n

    ii

    Review of Linear Algebra (Cont)

    Parseval identity for

    in subsp genrsquod by o n basis

    bull Pythagorean theorem

    bull ldquoDecomposition of Energyrdquo

    bull ANOVA - sums of squares

    x

    nvv 1

    2

    1

    22

    1

    2 aavxx

    n

    ii

    n

    ii

    Review of Linear Algebra (Cont)

    Parseval identity for

    in subsp genrsquod by o n basis

    bull Pythagorean theorem

    bull ldquoDecomposition of Energyrdquo

    bull ANOVA - sums of squares

    bull Transform has same length as

    ie ldquorotation in rdquo

    x

    nvv 1

    2

    1

    22

    1

    2 aavxx

    n

    ii

    n

    ii

    a xd

    Projection of a Vector onto a Subspace

    bull Idea Member of that is Closest to

    (ie ldquoBest Approxrsquonrdquo)

    Review of Linear Algebra (Cont)x

    xV

    V

    Projection of a Vector onto a Subspace

    bull Idea Member of that is Closest to

    (ie ldquoBest Approxrsquonrdquo)

    bull Find that Solves

    (ldquoLeast Squaresrdquo)

    Review of Linear Algebra (Cont)x

    xV

    V

    VxPV vxVv

    min

    Projection of a Vector onto a Subspace

    bull Idea Member of that is Closest to

    (ie ldquoBest Approxrsquonrdquo)

    bull Find that Solves

    (ldquoLeast Squaresrdquo)

    bull For Inner Product (Hilbert) Space

    Exists and is Unique

    Review of Linear Algebra (Cont)x

    xV

    V

    VxPV vxVv

    min

    xPV

    Projection of a Vector onto a Subspace (cont)

    bull General Solution in for Basis Matrix

    Review of Linear Algebra (Cont)

    d VB

    xBBBBxP tVV

    tVVV

    1

    Projection of a Vector onto a Subspace (cont)

    bull General Solution in for Basis Matrix

    bull So Projrsquon Operator is Matrix Multrsquon

    (thus projection is another linear operation)

    Review of Linear Algebra (Cont)

    d VB

    xBBBBxP tVV

    tVVV

    1

    tVV

    tVVV BBBBP

    1

    Projection of a Vector onto a Subspace (cont)

    bull General Solution in for Basis Matrix

    bull So Projrsquon Operator is Matrix Multrsquon

    (thus projection is another linear operation)

    (note same operation underlies least squares)

    Review of Linear Algebra (Cont)

    d VB

    xBBBBxP tVV

    tVVV

    1

    tVV

    tVVV BBBBP

    1

    Review of Linear Algebra (Cont)

    Projection using Orthonormal Basis

    bull Basis Matrix is Orthonormal

    nnVtV IBB

    10

    01

    1

    111

    1

    1

    nnn

    n

    ntn

    t

    vvvv

    vvvv

    vv

    v

    v

    nvv 1

    Review of Linear Algebra (Cont)

    Projection using Orthonormal Basis

    bull Basis Matrix is Orthonormal

    bull So =

    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

    (Recall Right Multrsquon)

    nnVtV IBB

    10

    01

    1

    111

    1

    1

    nnn

    n

    ntn

    t

    vvvv

    vvvv

    vv

    v

    v

    xBBxP tVVV

    x V

    nvv 1

    Review of Linear Algebra (Cont)

    Projection using Orthonormal Basis (cont)

    bull For Orthogonal Complement

    and

    V

    xPxPx VV 222xPxPx VV

    Review of Linear Algebra (Cont)

    Projection using Orthonormal Basis (cont)

    bull For Orthogonal Complement

    and

    bull Parseval Inequality

    V

    xPxPx VV 222xPxPx VV

    2

    1

    22

    1

    22 aavxxxP

    n

    ii

    n

    iiV

    Review of Linear Algebra (Cont)

    (Real) Unitary Matrices with

    bull Orthonormal Basis Matrix

    (So All of Above Applies)

    ddU IUU t

    Review of Linear Algebra (Cont)

    (Real) Unitary Matrices with

    bull Orthonormal Basis Matrix

    (So All of Above Applies)

    bull Note Transformrsquon is Distance Preserving

    ddU IUU t

    yxdyxyxyxUyUxUdn

    i ii 2

    1

    Review of Linear Algebra (Cont)

    (Real) Unitary Matrices with

    bull Orthonormal Basis Matrix

    (So All of Above Applies)

    bull Note Transformrsquon is Distance Preserving

    bull Lin Trans (Mult by ) is ~ Rotation

    bull But also Includes ldquoMirror Imagesrdquo

    ddU IUU t

    yxdyxyxyxUyUxUdn

    i ii 2

    1

    U

    Review of Linear Algebra (Cont)

    Singular Value Decomposition (SVD)

    For a Matrix

    Find

    ndX

    Review of Linear Algebra (Cont)

    Singular Value Decomposition (SVD)

    For a Matrix

    Find a Diagonal Matrix

    with Entries

    called Singular Values

    ndX

    ndS

    )min(1 ndss

    Review of Linear Algebra (Cont)

    Singular Value Decomposition (SVD)

    For a Matrix

    Find a Diagonal Matrix

    with Entries

    called Singular Values

    And Unitary (Rotation) Matrices

    (recall )

    ndX

    ndS

    )min(1 ndss

    ddU nnV

    IVVUU tt

    Review of Linear Algebra (Cont)

    Singular Value Decomposition (SVD)

    For a Matrix

    Find a Diagonal Matrix

    with Entries

    called Singular Values

    And Unitary (Rotation) Matrices

    (recall )

    So That

    ndX

    ndS

    )min(1 ndss

    ddU nnV

    IVVUU tt tUSVX

    Review of Linear Algebra (Cont)

    Intuition behind Singular Value Decomposition

    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

    vVSUvVSUvX tt

    Review of Linear Algebra (Cont)

    Intuition behind Singular Value Decomposition

    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

    bull First rotate

    X

    vVSUvVSUvX tt

    Review of Linear Algebra (Cont)

    Intuition behind Singular Value Decomposition

    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

    bull First rotate

    bull Second rescale coordinate axes (by )

    X

    vVSUvVSUvX tt

    is

    Review of Linear Algebra (Cont)

    Intuition behind Singular Value Decomposition

    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

    bull First rotate

    bull Second rescale coordinate axes (by )

    bull Third rotate again

    X

    vVSUvVSUvX tt

    is

    Review of Linear Algebra (Cont)

    Intuition behind Singular Value Decomposition

    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

    bull First rotate

    bull Second rescale coordinate axes (by )

    bull Third rotate again

    bull ie have diagonalized the transformation

    X

    vVSUvVSUvX tt

    is

    SVD Compact Representation

    Useful Labeling

    Singular Values in Increasing Order

    Review of Linear Algebra (Cont)

    )min(1 dnss

    SVD Compact Representation

    Useful Labeling

    Singular Values in Increasing Order

    Note singular values = 0 can be omitted

    (Since do ldquo0-Stretchingrdquo)

    Review of Linear Algebra (Cont)

    )min(1 dnss

    Review of Linear Algebra (Cont)

    r

    SVD Compact Representation

    Useful Labeling

    Singular Values in Increasing Order

    Note singular values = 0 can be omitted

    Let = of positive singular values

    )min(1 dnss

    Review of Linear Algebra (Cont)

    r

    SVD Compact Representation

    Useful Labeling

    Singular Values in Increasing Order

    Note singular values = 0 can be omitted

    Let = of positive singular values

    Then

    Where are truncations of

    trnrrrd VSUX

    VSU

    )min(1 dnss

    Review of Linear Algebra (Cont)

    SVD Full Representation

    =

    Graphics Display Assumes

    ndX ddU ndS nn

    tV

    nd

    Review of Linear Algebra (Cont)

    SVD Full Representation

    =

    Full Rank Basis Matrix

    ndX ddU ndS nn

    tV

    Review of Linear Algebra (Cont)

    SVD Full Representation

    =

    Full Rank Basis Matrix

    All 0s in Bottom

    ndX ddU ndS nn

    tV

    Review of Linear Algebra (Cont)

    SVD Reduced Representation

    =

    These Columns Get 0ed Out

    ndX ddU nnS nn

    tV

    nnd 0

    Review of Linear Algebra (Cont)

    SVD Reduced Representation

    =ndX ndU nnS nn

    tV

    Review of Linear Algebra (Cont)

    SVD Reduced Representation

    =

    Also Some of These May be 0

    ndX ndU nnS nn

    tV

    Review of Linear Algebra (Cont)

    SVD Compact Representation

    = ndX rdU

    rrS nrtV

    0

    Review of Linear Algebra (Cont)

    SVD Compact Representation

    =

    These Get 0ed Out

    ndX rdU

    rrS nrtV

    0

    Review of Linear Algebra (Cont)

    SVD Compact Representation

    = ndX rdU

    rrS nrtV

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition

    For a (Symmetric) Square Matrix

    Find

    ddX

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition

    For a (Symmetric) Square Matrix

    Find a Diagonal Matrix

    Called Eigenvalues

    Convenient Ordering

    ddX

    d

    D

    0

    01

    n 1

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition

    For a (Symmetric) Square Matrix

    Find a Diagonal Matrix

    And an Orthonormal Matrix

    (ie )

    ddX

    d

    D

    0

    01

    ddB

    ddtt IBBBB

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition

    For a (Symmetric) Square Matrix

    Find a Diagonal Matrix

    And an Orthonormal Matrix

    (ie )

    So that ie

    ddX

    d

    D

    0

    01

    ddB

    ddtt IBBBB

    DBBX tBDBX

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

    (looks similar)

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

    Complex (uses )

    VU

    1i

    Review of Linear Algebra (Cont)

    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

    VU

    X

    BVU

    1i

    • Participant Presentations
    • PCA to find clusters
    • Statistical Smoothing
    • Kernel Density Estimation
    • Scatterplot Smoothing
    • Statistical Smoothing (2)
    • SiZer Background
    • SiZer Background (2)
    • SiZer Background (3)
    • SiZer Background (4)
    • SiZer Background (5)
    • SiZer Overview
    • PCA to find clusters (2)
    • PCA to find clusters (3)
    • PCA to find clusters (4)
    • PCA to find clusters (5)
    • PCA to find clusters (6)
    • PCA to find clusters (7)
    • Recall Yeast Cell Cycle Data
    • Yeast Cell Cycle Data FDA View
    • Yeast Cell Cycle Data FDA View (2)
    • Yeast Cell Cycle Data FDA View (3)
    • Yeast Cell Cycles Freq 2 Proj
    • Frequency 2 Analysis
    • Frequency 2 Analysis (2)
    • Frequency 2 Analysis (3)
    • Yeast Cell Cycle
    • SiZer Study of Distrsquon of Angles
    • Reclassification of Major Genes
    • Compare to Previous Classifrsquon
    • New Subpopulation View
    • New Subpopulation View (2)
    • Detailed Look at PCA
    • PCA Rediscovery ndash Renaming
    • PCA Rediscovery ndash Renaming (2)
    • PCA Rediscovery ndash Renaming (3)
    • PCA Rediscovery ndash Renaming (4)
    • PCA Rediscovery ndash Renaming (5)
    • An Interesting Historical Note
    • An Interesting Historical Note (2)
    • Detailed Look at PCA (2)
    • Detailed Look at PCA (3)
    • Review of Linear Algebra
    • Review of Linear Algebra (2)
    • Review of Linear Algebra (3)
    • Review of Linear Algebra (Cont)
    • Review of Linear Algebra (Cont) (2)
    • Review of Linear Algebra (Cont) (3)
    • Review of Linear Algebra (Cont) (4)
    • Review of Linear Algebra (Cont) (5)
    • Review of Linear Algebra (Cont) (6)
    • Review of Linear Algebra (Cont) (7)
    • Review of Linear Algebra (Cont) (8)
    • Review of Linear Algebra (Cont) (9)
    • Review of Linear Algebra (Cont) (10)
    • Review of Linear Algebra (Cont) (11)
    • Review of Linear Algebra (Cont) (12)
    • Review of Linear Algebra (Cont) (13)
    • Review of Linear Algebra (Cont) (14)
    • Review of Linear Algebra (Cont) (15)
    • Review of Linear Algebra (Cont) (16)
    • Review of Linear Algebra (Cont) (17)
    • Review of Linear Algebra (Cont) (18)
    • Review of Linear Algebra (Cont) (19)
    • Review of Linear Algebra (Cont) (20)
    • Review of Linear Algebra (Cont) (21)
    • Review of Linear Algebra (Cont) (22)
    • Review of Linear Algebra (Cont) (23)
    • Review of Linear Algebra (Cont) (24)
    • Review of Linear Algebra (Cont) (25)
    • Review of Linear Algebra (Cont) (26)
    • Review of Linear Algebra (Cont) (27)
    • Review of Linear Algebra (Cont) (28)
    • Review of Linear Algebra (Cont) (29)
    • Review of Linear Algebra (Cont) (30)
    • Review of Linear Algebra (Cont) (31)
    • Review of Linear Algebra (Cont) (32)
    • Review of Linear Algebra (Cont) (33)
    • Review of Linear Algebra (Cont) (34)
    • Review of Linear Algebra (Cont) (35)
    • Review of Linear Algebra (Cont) (36)
    • Review of Linear Algebra (Cont) (37)
    • Review of Linear Algebra (Cont) (38)
    • Review of Linear Algebra (Cont) (39)
    • Review of Linear Algebra (Cont) (40)
    • Review of Linear Algebra (Cont) (41)
    • Review of Linear Algebra (Cont) (42)
    • Review of Linear Algebra (Cont) (43)
    • Review of Linear Algebra (Cont) (44)
    • Review of Linear Algebra (Cont) (45)
    • Review of Linear Algebra (Cont) (46)
    • Review of Linear Algebra (Cont) (47)
    • Review of Linear Algebra (Cont) (48)
    • Review of Linear Algebra (Cont) (49)
    • Review of Linear Algebra (Cont) (50)
    • Review of Linear Algebra (Cont) (51)
    • Review of Linear Algebra (Cont) (52)
    • Review of Linear Algebra (Cont) (53)
    • Review of Linear Algebra (Cont) (54)
    • Review of Linear Algebra (Cont) (55)
    • Review of Linear Algebra (Cont) (56)
    • Review of Linear Algebra (Cont) (57)
    • Review of Linear Algebra (Cont) (58)
    • Review of Linear Algebra (Cont) (59)
    • Review of Linear Algebra (Cont) (60)
    • Review of Linear Algebra (Cont) (61)
    • Review of Linear Algebra (Cont) (62)
    • Review of Linear Algebra (Cont) (63)
    • Review of Linear Algebra (Cont) (64)
    • Review of Linear Algebra (Cont) (65)
    • Review of Linear Algebra (Cont) (66)
    • Review of Linear Algebra (Cont) (67)
    • Review of Linear Algebra (Cont) (68)
    • Review of Linear Algebra (Cont) (69)
    • Review of Linear Algebra (Cont) (70)
    • Review of Linear Algebra (Cont) (71)
    • Review of Linear Algebra (Cont) (72)
    • Review of Linear Algebra (Cont) (73)
    • Review of Linear Algebra (Cont) (74)
    • Review of Linear Algebra (Cont) (75)
    • Review of Linear Algebra (Cont) (76)
    • Review of Linear Algebra (Cont) (77)
    • Review of Linear Algebra (Cont) (78)
    • Review of Linear Algebra (Cont) (79)
    • Review of Linear Algebra (Cont) (80)
    • Review of Linear Algebra (Cont) (81)

      Statistical SmoothingIn 1 Dimension 2 Major Settings

      bull Density EstimationldquoHistogramsrdquo

      bull Nonparametric RegressionldquoScatterplot Smoothingrdquo

      Kernel Density EstimationChondrite Databull Sum pieces to estimate densitybull Suggests 3 modes (rock sources)

      Scatterplot SmoothingEg Bralower Fossil Data ndash some

      smooths

      Statistical SmoothingFundamental Question

      For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

      Smoothingrdquo

      Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

      SiZer BackgroundFun Scale Space Views (Incomes Data)

      Surface View

      SiZer BackgroundSiZer analysis of British Incomes data

      >

      SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

      statistically significant

      Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

      wiggles not significant

      Again all are correct

      just different scales

      SiZer BackgroundScale Space and Kernel Choice

      ie Shape of Window

      Compelling Answer Gaussian

      Only ldquoVariation Diminishingrdquo Kernel Shape

      I e Modes decreases with bandwidth h

      Lindebergh (1994)

      Chaudhuri amp Marron (2000)

      SiZer BackgroundRecall

      Hidalgo

      Stamps

      Data

      >

      SiZer Overview

      Would you like to try smoothing amp

      SiZer

      bull Marron Software Website as Before

      bull In ldquoSmoothingrdquo Directory

      ndash kdeSMm

      ndash nprSMm

      ndash sizerSMm

      bull Recall ldquogtgt help sizerSMrdquo for

      usage

      PCA to find clustersReturn to PCA of Mass Flux Data

      PCA to find clustersSiZer analysis of Mass Flux PC1

      PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

      PCA to find clustersSiZer analysis of Mass Flux PC1

      Also in Curvature

      PCA to find clustersSiZer analysis of Mass Flux PC1

      And in Other Comprsquos

      PCA to find clustersSiZer analysis of Mass Flux PC1

      Conclusion

      bull Found 3 significant clusters

      bull Correspond to 3 known ldquocloud typesrdquo

      bull Worth deeper investigation

      Recall Yeast Cell Cycle Data

      bull ldquoGene Expressionrdquo ndash Micro-array data

      bull Data (after major preprocessing) Expression ldquolevelrdquo of

      bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

      10s)bull Interesting statistical issue

      High Dimension Low Sample Size data

      (HDLSS)

      Yeast Cell Cycle Data FDA View

      Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

      Yeast Cell Cycle Data FDA View

      Periodic genes

      Naiumlve

      approach

      Simple PCA

      Yeast Cell Cycle Data FDA Viewbull Central question which genes are

      ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

      ldquovariationrdquobull PCA finds ldquodirections of maximal

      variationrdquobull Often but not always same as

      ldquointeresting directionsrdquobull Here need better approach to study

      periodicities

      Yeast Cell Cycles Freq 2 Proj

      PCA on

      Freq 2

      Periodic

      Component

      Of Data

      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

      Approach from Zhao Marron amp Wells (2004)

      Frequency 2 Analysis

      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

      classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

      notable differencesbull Now try to improve ldquophase classificationrdquo

      Yeast Cell CycleRevisit ldquophase classificationrdquo

      approachbull Use outer 200 genes

      (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

      (finds significant bumps etc in histogram)

      bull Carefully redrew boundariesbull Check by studying kde angles

      SiZer Study of Distrsquon of Angles

      Reclassification of Major Genes

      Compare to Previous Classifrsquon

      New Subpopulation View

      New Subpopulation View

      NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

      Detailed Look at PCA

      Now Study ldquoFolklorerdquo More Carefully

      bull BackGround

      bull History

      bull Underpinnings

      (Mathematical amp Computational)

      Good Overall Reference Jolliffe (2002)

      PCA Rediscovery ndash Renaming

      Statistics Principal Component Analysis (PCA)

      PCA Rediscovery ndash Renaming

      Statistics Principal Component Analysis (PCA)

      Social Sciences Factor Analysis (PCA is a subset)

      PCA Rediscovery ndash Renaming

      Statistics Principal Component Analysis (PCA)

      Social Sciences Factor Analysis (PCA is a subset)

      Probability Electrical EngKarhunen ndash Loeve expansion

      PCA Rediscovery ndash Renaming

      Statistics Principal Component Analysis (PCA)

      Social Sciences Factor Analysis (PCA is a subset)

      Probability Electrical EngKarhunen ndash Loeve expansion

      Applied MathematicsProper Orthogonal Decomposition (POD)

      PCA Rediscovery ndash Renaming

      Statistics Principal Component Analysis (PCA)

      Social Sciences Factor Analysis (PCA is a subset)

      Probability Electrical EngKarhunen ndash Loeve expansion

      Applied MathematicsProper Orthogonal Decomposition (POD)

      Geo-Sciences Empirical Orthogonal Functions (EOF)

      An Interesting Historical Note

      The 1st () application of PCA to Functional

      Data Analysis

      An Interesting Historical Note

      The 1st () application of PCA to Functional

      Data Analysis

      Rao (1958)

      1st Paper with ldquoCurves as Data Objectsrdquo

      viewpoint

      Detailed Look at PCA

      Three Important (amp Interesting) Viewpoints

      1 Mathematics

      2 Numerics

      3 Statistics

      Goal Study Interrelationships

      Detailed Look at PCA

      Three Important (amp Interesting) Viewpoints

      1 Mathematics

      2 Numerics

      3 Statistics

      1st Review Linear Alg and Multivar Prob

      Review of Linear Algebra

      Vector Space

      bull set of ldquovectorsrdquo

      bull and ldquoscalarsrdquo (coefficients)

      xa

      Review of Linear Algebra

      Vector Space

      bull set of ldquovectorsrdquo

      bull and ldquoscalarsrdquo (coefficients)

      bull ldquoclosedrdquo under ldquolinear combinationrdquo

      ( in space)

      xa

      i

      ii xa

      Review of Linear Algebra

      Vector Space

      bull set of ldquovectorsrdquo

      bull and ldquoscalarsrdquo (coefficients)

      bull ldquoclosedrdquo under ldquolinear combinationrdquo

      ( in space)

      eg

      ldquo dim Euclidrsquon spacerdquo

      xa

      i

      ii xa

      d

      d

      d xx

      x

      x

      x 1

      1

      d

      Review of Linear Algebra (Cont)

      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

      Review of Linear Algebra (Cont)

      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

      Review of Linear Algebra (Cont)

      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

      Note Planes not Through the Origin

      are not Subspaces

      (Do not Contain )00 x

      Review of Linear Algebra (Cont)

      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

      (all Linear Combos of them =

      = Containing Hyperplane

      through Origin)

      Review of Linear Algebra (Cont)

      Basis of Subspace Set of Vectors that

      bull Span ie Everything is a Lin Com of them

      bull are Linearly Indeprsquot ie Lin Com is Unique

      Review of Linear Algebra (Cont)

      Basis of Subspace Set of Vectors that

      bull Span ie Everything is a Lin Com of them

      bull are Linearly Indeprsquot ie Lin Com is Unique

      bull eg ldquoUnit Vector Basisrdquo

      bull Since

      d

      1

      0

      0

      0

      1

      0

      0

      0

      1

      1

      0

      0

      0

      1

      0

      0

      0

      1

      212

      1

      d

      d

      xxx

      x

      x

      x

      Review of Linear Algebra (Cont)

      Basis Matrix of subspace of

      Given a basis

      create matrix of columns

      dnvv 1

      nddnd

      n

      n

      vv

      vv

      vvB

      1

      111

      1

      Review of Linear Algebra (Cont)

      Then linear combo is a matrix multiplicatrsquon

      where

      n

      iii aBva

      1

      na

      a

      a 1

      Review of Linear Algebra (Cont)

      Then linear combo is a matrix multiplicatrsquon

      where

      Note Right Multiplication Gives

      Linear Combination of Column Vectors

      n

      iii aBva

      1

      na

      a

      a 1

      Review of Linear Algebra (Cont)

      Then linear combo is a matrix multiplicatrsquon

      where

      Check sizes

      n

      iii aBva

      1

      na

      a

      a 1

      )1()(1 nndd

      Review of Linear Algebra (Cont)

      Aside on Matrix Multiplication (linear transformatrsquon)

      For matrices

      Define the Matrix Product

      mkk

      m

      aa

      aa

      A

      1

      111

      nmm

      n

      bb

      bb

      B

      1

      111

      m

      iniik

      m

      iiik

      m

      inii

      m

      iii

      baba

      baba

      AB

      1

      11

      11

      111

      Review of Linear Algebra (Cont)

      Aside on Matrix Multiplication (linear transformatrsquon)

      For matrices

      Define the Matrix Product

      (Inner Products of Rows With Columns )

      mkk

      m

      aa

      aa

      A

      1

      111

      nmm

      n

      bb

      bb

      B

      1

      111

      m

      iniik

      m

      iiik

      m

      inii

      m

      iii

      baba

      baba

      AB

      1

      11

      11

      111

      A B

      Review of Linear Algebra (Cont)

      Aside on Matrix Multiplication (linear transformatrsquon)

      For matrices

      Define the Matrix Product

      (Inner Products of Rows With Columns )

      (Composition of Linear Transformations)

      mkk

      m

      aa

      aa

      A

      1

      111

      nmm

      n

      bb

      bb

      B

      1

      111

      m

      iniik

      m

      iiik

      m

      inii

      m

      iii

      baba

      baba

      AB

      1

      11

      11

      111

      A B

      Review of Linear Algebra (Cont)

      Aside on Matrix Multiplication (linear transformatrsquon)

      For matrices

      Define the Matrix Product

      (Inner Products of Rows With Columns )

      (Composition of Linear Transformations)

      Often Useful to Check Sizes

      mkk

      m

      aa

      aa

      A

      1

      111

      nmm

      n

      bb

      bb

      B

      1

      111

      m

      iniik

      m

      iiik

      m

      inii

      m

      iii

      baba

      baba

      AB

      1

      11

      11

      111

      nmmknk

      A B

      Review of Linear Algebra (Cont)

      Aside on Matrix Multiplication (linear transformatrsquon)

      For matrices

      Define the Matrix Product

      (Inner Products of Rows With Columns )

      (Composition of Linear Transformations)

      Often Useful to Check Sizes

      mkk

      m

      aa

      aa

      A

      1

      111

      nmm

      n

      bb

      bb

      B

      1

      111

      m

      iniik

      m

      iiik

      m

      inii

      m

      iii

      baba

      baba

      AB

      1

      11

      11

      111

      nmmknk

      A B

      Review of Linear Algebra (Cont)

      Matrix Trace

      bull For a Square Matrix

      bull Define

      mmm

      m

      aa

      aa

      A

      1

      111

      m

      iiiaAtr

      1)(

      Review of Linear Algebra (Cont)

      Matrix Trace

      bull For a Square Matrix

      bull Define

      bull Trace Commutes with Matrix Multiplication

      mmm

      m

      aa

      aa

      A

      1

      111

      m

      iiiaAtr

      1)(

      BAtrABtr

      Review of Linear Algebra (Cont)

      Dimension of Subspace (a Notion of ldquoSizerdquo)

      bull Number of Elements in a Basis (Unique)

      Review of Linear Algebra (Cont)

      Dimension of Subspace (a Notion of ldquoSizerdquo)

      bull Number of Elements in a Basis (Unique)

      bull (Use Basis Above)

      bull eg dim of a line is 1

      bull eg dim of a plane is 2

      dd dim

      Review of Linear Algebra (Cont)

      Dimension of Subspace (a Notion of ldquoSizerdquo)

      bull Number of Elements in a Basis (Unique)

      bull (Use Basis Above)

      bull eg dim of a line is 1

      bull eg dim of a plane is 2

      bull Dimension is ldquoDegrees of Freedomrdquo

      (in Statistical Uses eg ANOVA)

      dd dim

      Review of Linear Algebra (Cont)

      Norm of a Vector

      bull in d 21

      21

      1

      2 xxxx td

      jj

      Review of Linear Algebra (Cont)

      Norm of a Vector

      bull in

      bull Idea length of the vector

      d 2121

      1

      2 xxxx td

      jj

      Review of Linear Algebra (Cont)

      Norm of a Vector

      bull in

      bull Idea length of the vector

      bull Note strange properties for high

      eg ldquolength of diagonal of unit cuberdquo =

      d 2121

      1

      2 xxxx td

      jj

      d

      d

      Review of Linear Algebra (Cont)

      Norm of a Vector (cont)

      bull Length Normalized Vector

      (has Length 1 thus on Surf of Unit Sphere

      amp is a Direction Vector)

      x

      x

      Review of Linear Algebra (Cont)

      Norm of a Vector (cont)

      bull Length Normalized Vector

      (has Length 1 thus on Surf of Unit Sphere

      amp is a Direction Vector)

      bull Define Distance as

      x

      x

      yxyxyxyxd t

      Review of Linear Algebra (Cont)

      Inner (Dot Scalar) Product

      bull for Vectors and

      yxyxyx td

      jjj

      1

      x y

      Review of Linear Algebra (Cont)

      Inner (Dot Scalar) Product

      bull for Vectors and

      bull Related to Norm via

      yxyxyx td

      jjj

      1

      x y

      21 xxx

      Review of Linear Algebra (Cont)

      Inner (Dot Scalar) Product (cont)

      bull measures ldquoangle between and rdquo as

      yyxx

      yx

      yx

      yxyxangle

      tt

      t

      11 cos

      cos

      x y

      Review of Linear Algebra (Cont)

      Inner (Dot Scalar) Product (cont)

      bull measures ldquoangle between and rdquo as

      bull key to Orthogonality ie Perpendiculrsquoty

      if and only if

      yyxx

      yx

      yx

      yxyxangle

      tt

      t

      11 cos

      cos

      x y

      yx 0 yx

      Review of Linear Algebra (Cont)

      Orthonormal Basis

      bull All Orthogonal to each other

      ie for

      bull All have Length 1

      ie for

      nvv 1

      1 ii vv

      0 ii vv ii

      ni 1

      Review of Linear Algebra (Cont)

      Orthonormal Basis (cont)

      bull Spectral Representation

      where

      nvv 1

      n

      iii vax

      1

      ii vxa

      Review of Linear Algebra (Cont)

      Orthonormal Basis (cont)

      bull Spectral Representation

      where

      (Coefficient is Inner Product Cool Notation)

      nvv 1

      n

      iii vax

      1

      ii vxa

      Review of Linear Algebra (Cont)

      Orthonormal Basis (cont)

      bull Spectral Representation

      where

      Check

      nvv 1

      n

      iii vax

      1

      ii vxa

      iii

      n

      iii

      n

      iiii avvavvavx

      1

      1

      Review of Linear Algebra (Cont)

      Orthonormal Basis (cont)

      bull Spectral Representation

      where

      Check

      bull Matrix Notation where ie

      For the Basis Matrix

      nvv 1

      n

      iii vax

      1

      ii vxa

      iii

      n

      iii

      n

      iiii avvavvavx

      1

      1

      aBx Bxa tt xBa t

      nvvB 1

      Review of Linear Algebra (Cont)

      Orthonormal Basis (cont)

      bull Spectral Representation

      where

      Check

      bull Matrix Notation where ie

      is called transform of

      (eg Fourier or Wavelet)

      nvv 1

      n

      iii vax

      1

      ii vxa

      iii

      n

      iii

      n

      iiii avvavvavx

      1

      1

      aBx Bxa tt xBa t

      xa

      Review of Linear Algebra (Cont)

      Parseval identity for

      in subsp genrsquod by o n basis

      x

      nvv 1

      2

      1

      22

      1

      2 aavxx

      n

      ii

      n

      ii

      Review of Linear Algebra (Cont)

      Parseval identity for

      in subsp genrsquod by o n basis

      bull Pythagorean theorem

      bull ldquoDecomposition of Energyrdquo

      bull ANOVA - sums of squares

      x

      nvv 1

      2

      1

      22

      1

      2 aavxx

      n

      ii

      n

      ii

      Review of Linear Algebra (Cont)

      Parseval identity for

      in subsp genrsquod by o n basis

      bull Pythagorean theorem

      bull ldquoDecomposition of Energyrdquo

      bull ANOVA - sums of squares

      bull Transform has same length as

      ie ldquorotation in rdquo

      x

      nvv 1

      2

      1

      22

      1

      2 aavxx

      n

      ii

      n

      ii

      a xd

      Projection of a Vector onto a Subspace

      bull Idea Member of that is Closest to

      (ie ldquoBest Approxrsquonrdquo)

      Review of Linear Algebra (Cont)x

      xV

      V

      Projection of a Vector onto a Subspace

      bull Idea Member of that is Closest to

      (ie ldquoBest Approxrsquonrdquo)

      bull Find that Solves

      (ldquoLeast Squaresrdquo)

      Review of Linear Algebra (Cont)x

      xV

      V

      VxPV vxVv

      min

      Projection of a Vector onto a Subspace

      bull Idea Member of that is Closest to

      (ie ldquoBest Approxrsquonrdquo)

      bull Find that Solves

      (ldquoLeast Squaresrdquo)

      bull For Inner Product (Hilbert) Space

      Exists and is Unique

      Review of Linear Algebra (Cont)x

      xV

      V

      VxPV vxVv

      min

      xPV

      Projection of a Vector onto a Subspace (cont)

      bull General Solution in for Basis Matrix

      Review of Linear Algebra (Cont)

      d VB

      xBBBBxP tVV

      tVVV

      1

      Projection of a Vector onto a Subspace (cont)

      bull General Solution in for Basis Matrix

      bull So Projrsquon Operator is Matrix Multrsquon

      (thus projection is another linear operation)

      Review of Linear Algebra (Cont)

      d VB

      xBBBBxP tVV

      tVVV

      1

      tVV

      tVVV BBBBP

      1

      Projection of a Vector onto a Subspace (cont)

      bull General Solution in for Basis Matrix

      bull So Projrsquon Operator is Matrix Multrsquon

      (thus projection is another linear operation)

      (note same operation underlies least squares)

      Review of Linear Algebra (Cont)

      d VB

      xBBBBxP tVV

      tVVV

      1

      tVV

      tVVV BBBBP

      1

      Review of Linear Algebra (Cont)

      Projection using Orthonormal Basis

      bull Basis Matrix is Orthonormal

      nnVtV IBB

      10

      01

      1

      111

      1

      1

      nnn

      n

      ntn

      t

      vvvv

      vvvv

      vv

      v

      v

      nvv 1

      Review of Linear Algebra (Cont)

      Projection using Orthonormal Basis

      bull Basis Matrix is Orthonormal

      bull So =

      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

      (Recall Right Multrsquon)

      nnVtV IBB

      10

      01

      1

      111

      1

      1

      nnn

      n

      ntn

      t

      vvvv

      vvvv

      vv

      v

      v

      xBBxP tVVV

      x V

      nvv 1

      Review of Linear Algebra (Cont)

      Projection using Orthonormal Basis (cont)

      bull For Orthogonal Complement

      and

      V

      xPxPx VV 222xPxPx VV

      Review of Linear Algebra (Cont)

      Projection using Orthonormal Basis (cont)

      bull For Orthogonal Complement

      and

      bull Parseval Inequality

      V

      xPxPx VV 222xPxPx VV

      2

      1

      22

      1

      22 aavxxxP

      n

      ii

      n

      iiV

      Review of Linear Algebra (Cont)

      (Real) Unitary Matrices with

      bull Orthonormal Basis Matrix

      (So All of Above Applies)

      ddU IUU t

      Review of Linear Algebra (Cont)

      (Real) Unitary Matrices with

      bull Orthonormal Basis Matrix

      (So All of Above Applies)

      bull Note Transformrsquon is Distance Preserving

      ddU IUU t

      yxdyxyxyxUyUxUdn

      i ii 2

      1

      Review of Linear Algebra (Cont)

      (Real) Unitary Matrices with

      bull Orthonormal Basis Matrix

      (So All of Above Applies)

      bull Note Transformrsquon is Distance Preserving

      bull Lin Trans (Mult by ) is ~ Rotation

      bull But also Includes ldquoMirror Imagesrdquo

      ddU IUU t

      yxdyxyxyxUyUxUdn

      i ii 2

      1

      U

      Review of Linear Algebra (Cont)

      Singular Value Decomposition (SVD)

      For a Matrix

      Find

      ndX

      Review of Linear Algebra (Cont)

      Singular Value Decomposition (SVD)

      For a Matrix

      Find a Diagonal Matrix

      with Entries

      called Singular Values

      ndX

      ndS

      )min(1 ndss

      Review of Linear Algebra (Cont)

      Singular Value Decomposition (SVD)

      For a Matrix

      Find a Diagonal Matrix

      with Entries

      called Singular Values

      And Unitary (Rotation) Matrices

      (recall )

      ndX

      ndS

      )min(1 ndss

      ddU nnV

      IVVUU tt

      Review of Linear Algebra (Cont)

      Singular Value Decomposition (SVD)

      For a Matrix

      Find a Diagonal Matrix

      with Entries

      called Singular Values

      And Unitary (Rotation) Matrices

      (recall )

      So That

      ndX

      ndS

      )min(1 ndss

      ddU nnV

      IVVUU tt tUSVX

      Review of Linear Algebra (Cont)

      Intuition behind Singular Value Decomposition

      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

      vVSUvVSUvX tt

      Review of Linear Algebra (Cont)

      Intuition behind Singular Value Decomposition

      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

      bull First rotate

      X

      vVSUvVSUvX tt

      Review of Linear Algebra (Cont)

      Intuition behind Singular Value Decomposition

      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

      bull First rotate

      bull Second rescale coordinate axes (by )

      X

      vVSUvVSUvX tt

      is

      Review of Linear Algebra (Cont)

      Intuition behind Singular Value Decomposition

      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

      bull First rotate

      bull Second rescale coordinate axes (by )

      bull Third rotate again

      X

      vVSUvVSUvX tt

      is

      Review of Linear Algebra (Cont)

      Intuition behind Singular Value Decomposition

      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

      bull First rotate

      bull Second rescale coordinate axes (by )

      bull Third rotate again

      bull ie have diagonalized the transformation

      X

      vVSUvVSUvX tt

      is

      SVD Compact Representation

      Useful Labeling

      Singular Values in Increasing Order

      Review of Linear Algebra (Cont)

      )min(1 dnss

      SVD Compact Representation

      Useful Labeling

      Singular Values in Increasing Order

      Note singular values = 0 can be omitted

      (Since do ldquo0-Stretchingrdquo)

      Review of Linear Algebra (Cont)

      )min(1 dnss

      Review of Linear Algebra (Cont)

      r

      SVD Compact Representation

      Useful Labeling

      Singular Values in Increasing Order

      Note singular values = 0 can be omitted

      Let = of positive singular values

      )min(1 dnss

      Review of Linear Algebra (Cont)

      r

      SVD Compact Representation

      Useful Labeling

      Singular Values in Increasing Order

      Note singular values = 0 can be omitted

      Let = of positive singular values

      Then

      Where are truncations of

      trnrrrd VSUX

      VSU

      )min(1 dnss

      Review of Linear Algebra (Cont)

      SVD Full Representation

      =

      Graphics Display Assumes

      ndX ddU ndS nn

      tV

      nd

      Review of Linear Algebra (Cont)

      SVD Full Representation

      =

      Full Rank Basis Matrix

      ndX ddU ndS nn

      tV

      Review of Linear Algebra (Cont)

      SVD Full Representation

      =

      Full Rank Basis Matrix

      All 0s in Bottom

      ndX ddU ndS nn

      tV

      Review of Linear Algebra (Cont)

      SVD Reduced Representation

      =

      These Columns Get 0ed Out

      ndX ddU nnS nn

      tV

      nnd 0

      Review of Linear Algebra (Cont)

      SVD Reduced Representation

      =ndX ndU nnS nn

      tV

      Review of Linear Algebra (Cont)

      SVD Reduced Representation

      =

      Also Some of These May be 0

      ndX ndU nnS nn

      tV

      Review of Linear Algebra (Cont)

      SVD Compact Representation

      = ndX rdU

      rrS nrtV

      0

      Review of Linear Algebra (Cont)

      SVD Compact Representation

      =

      These Get 0ed Out

      ndX rdU

      rrS nrtV

      0

      Review of Linear Algebra (Cont)

      SVD Compact Representation

      = ndX rdU

      rrS nrtV

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition

      For a (Symmetric) Square Matrix

      Find

      ddX

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition

      For a (Symmetric) Square Matrix

      Find a Diagonal Matrix

      Called Eigenvalues

      Convenient Ordering

      ddX

      d

      D

      0

      01

      n 1

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition

      For a (Symmetric) Square Matrix

      Find a Diagonal Matrix

      And an Orthonormal Matrix

      (ie )

      ddX

      d

      D

      0

      01

      ddB

      ddtt IBBBB

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition

      For a (Symmetric) Square Matrix

      Find a Diagonal Matrix

      And an Orthonormal Matrix

      (ie )

      So that ie

      ddX

      d

      D

      0

      01

      ddB

      ddtt IBBBB

      DBBX tBDBX

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

      (looks similar)

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

      Complex (uses )

      VU

      1i

      Review of Linear Algebra (Cont)

      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

      VU

      X

      BVU

      1i

      • Participant Presentations
      • PCA to find clusters
      • Statistical Smoothing
      • Kernel Density Estimation
      • Scatterplot Smoothing
      • Statistical Smoothing (2)
      • SiZer Background
      • SiZer Background (2)
      • SiZer Background (3)
      • SiZer Background (4)
      • SiZer Background (5)
      • SiZer Overview
      • PCA to find clusters (2)
      • PCA to find clusters (3)
      • PCA to find clusters (4)
      • PCA to find clusters (5)
      • PCA to find clusters (6)
      • PCA to find clusters (7)
      • Recall Yeast Cell Cycle Data
      • Yeast Cell Cycle Data FDA View
      • Yeast Cell Cycle Data FDA View (2)
      • Yeast Cell Cycle Data FDA View (3)
      • Yeast Cell Cycles Freq 2 Proj
      • Frequency 2 Analysis
      • Frequency 2 Analysis (2)
      • Frequency 2 Analysis (3)
      • Yeast Cell Cycle
      • SiZer Study of Distrsquon of Angles
      • Reclassification of Major Genes
      • Compare to Previous Classifrsquon
      • New Subpopulation View
      • New Subpopulation View (2)
      • Detailed Look at PCA
      • PCA Rediscovery ndash Renaming
      • PCA Rediscovery ndash Renaming (2)
      • PCA Rediscovery ndash Renaming (3)
      • PCA Rediscovery ndash Renaming (4)
      • PCA Rediscovery ndash Renaming (5)
      • An Interesting Historical Note
      • An Interesting Historical Note (2)
      • Detailed Look at PCA (2)
      • Detailed Look at PCA (3)
      • Review of Linear Algebra
      • Review of Linear Algebra (2)
      • Review of Linear Algebra (3)
      • Review of Linear Algebra (Cont)
      • Review of Linear Algebra (Cont) (2)
      • Review of Linear Algebra (Cont) (3)
      • Review of Linear Algebra (Cont) (4)
      • Review of Linear Algebra (Cont) (5)
      • Review of Linear Algebra (Cont) (6)
      • Review of Linear Algebra (Cont) (7)
      • Review of Linear Algebra (Cont) (8)
      • Review of Linear Algebra (Cont) (9)
      • Review of Linear Algebra (Cont) (10)
      • Review of Linear Algebra (Cont) (11)
      • Review of Linear Algebra (Cont) (12)
      • Review of Linear Algebra (Cont) (13)
      • Review of Linear Algebra (Cont) (14)
      • Review of Linear Algebra (Cont) (15)
      • Review of Linear Algebra (Cont) (16)
      • Review of Linear Algebra (Cont) (17)
      • Review of Linear Algebra (Cont) (18)
      • Review of Linear Algebra (Cont) (19)
      • Review of Linear Algebra (Cont) (20)
      • Review of Linear Algebra (Cont) (21)
      • Review of Linear Algebra (Cont) (22)
      • Review of Linear Algebra (Cont) (23)
      • Review of Linear Algebra (Cont) (24)
      • Review of Linear Algebra (Cont) (25)
      • Review of Linear Algebra (Cont) (26)
      • Review of Linear Algebra (Cont) (27)
      • Review of Linear Algebra (Cont) (28)
      • Review of Linear Algebra (Cont) (29)
      • Review of Linear Algebra (Cont) (30)
      • Review of Linear Algebra (Cont) (31)
      • Review of Linear Algebra (Cont) (32)
      • Review of Linear Algebra (Cont) (33)
      • Review of Linear Algebra (Cont) (34)
      • Review of Linear Algebra (Cont) (35)
      • Review of Linear Algebra (Cont) (36)
      • Review of Linear Algebra (Cont) (37)
      • Review of Linear Algebra (Cont) (38)
      • Review of Linear Algebra (Cont) (39)
      • Review of Linear Algebra (Cont) (40)
      • Review of Linear Algebra (Cont) (41)
      • Review of Linear Algebra (Cont) (42)
      • Review of Linear Algebra (Cont) (43)
      • Review of Linear Algebra (Cont) (44)
      • Review of Linear Algebra (Cont) (45)
      • Review of Linear Algebra (Cont) (46)
      • Review of Linear Algebra (Cont) (47)
      • Review of Linear Algebra (Cont) (48)
      • Review of Linear Algebra (Cont) (49)
      • Review of Linear Algebra (Cont) (50)
      • Review of Linear Algebra (Cont) (51)
      • Review of Linear Algebra (Cont) (52)
      • Review of Linear Algebra (Cont) (53)
      • Review of Linear Algebra (Cont) (54)
      • Review of Linear Algebra (Cont) (55)
      • Review of Linear Algebra (Cont) (56)
      • Review of Linear Algebra (Cont) (57)
      • Review of Linear Algebra (Cont) (58)
      • Review of Linear Algebra (Cont) (59)
      • Review of Linear Algebra (Cont) (60)
      • Review of Linear Algebra (Cont) (61)
      • Review of Linear Algebra (Cont) (62)
      • Review of Linear Algebra (Cont) (63)
      • Review of Linear Algebra (Cont) (64)
      • Review of Linear Algebra (Cont) (65)
      • Review of Linear Algebra (Cont) (66)
      • Review of Linear Algebra (Cont) (67)
      • Review of Linear Algebra (Cont) (68)
      • Review of Linear Algebra (Cont) (69)
      • Review of Linear Algebra (Cont) (70)
      • Review of Linear Algebra (Cont) (71)
      • Review of Linear Algebra (Cont) (72)
      • Review of Linear Algebra (Cont) (73)
      • Review of Linear Algebra (Cont) (74)
      • Review of Linear Algebra (Cont) (75)
      • Review of Linear Algebra (Cont) (76)
      • Review of Linear Algebra (Cont) (77)
      • Review of Linear Algebra (Cont) (78)
      • Review of Linear Algebra (Cont) (79)
      • Review of Linear Algebra (Cont) (80)
      • Review of Linear Algebra (Cont) (81)

        Kernel Density EstimationChondrite Databull Sum pieces to estimate densitybull Suggests 3 modes (rock sources)

        Scatterplot SmoothingEg Bralower Fossil Data ndash some

        smooths

        Statistical SmoothingFundamental Question

        For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

        Smoothingrdquo

        Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

        SiZer BackgroundFun Scale Space Views (Incomes Data)

        Surface View

        SiZer BackgroundSiZer analysis of British Incomes data

        >

        SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

        statistically significant

        Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

        wiggles not significant

        Again all are correct

        just different scales

        SiZer BackgroundScale Space and Kernel Choice

        ie Shape of Window

        Compelling Answer Gaussian

        Only ldquoVariation Diminishingrdquo Kernel Shape

        I e Modes decreases with bandwidth h

        Lindebergh (1994)

        Chaudhuri amp Marron (2000)

        SiZer BackgroundRecall

        Hidalgo

        Stamps

        Data

        >

        SiZer Overview

        Would you like to try smoothing amp

        SiZer

        bull Marron Software Website as Before

        bull In ldquoSmoothingrdquo Directory

        ndash kdeSMm

        ndash nprSMm

        ndash sizerSMm

        bull Recall ldquogtgt help sizerSMrdquo for

        usage

        PCA to find clustersReturn to PCA of Mass Flux Data

        PCA to find clustersSiZer analysis of Mass Flux PC1

        PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

        PCA to find clustersSiZer analysis of Mass Flux PC1

        Also in Curvature

        PCA to find clustersSiZer analysis of Mass Flux PC1

        And in Other Comprsquos

        PCA to find clustersSiZer analysis of Mass Flux PC1

        Conclusion

        bull Found 3 significant clusters

        bull Correspond to 3 known ldquocloud typesrdquo

        bull Worth deeper investigation

        Recall Yeast Cell Cycle Data

        bull ldquoGene Expressionrdquo ndash Micro-array data

        bull Data (after major preprocessing) Expression ldquolevelrdquo of

        bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

        10s)bull Interesting statistical issue

        High Dimension Low Sample Size data

        (HDLSS)

        Yeast Cell Cycle Data FDA View

        Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

        Yeast Cell Cycle Data FDA View

        Periodic genes

        Naiumlve

        approach

        Simple PCA

        Yeast Cell Cycle Data FDA Viewbull Central question which genes are

        ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

        ldquovariationrdquobull PCA finds ldquodirections of maximal

        variationrdquobull Often but not always same as

        ldquointeresting directionsrdquobull Here need better approach to study

        periodicities

        Yeast Cell Cycles Freq 2 Proj

        PCA on

        Freq 2

        Periodic

        Component

        Of Data

        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

        Approach from Zhao Marron amp Wells (2004)

        Frequency 2 Analysis

        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

        classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

        notable differencesbull Now try to improve ldquophase classificationrdquo

        Yeast Cell CycleRevisit ldquophase classificationrdquo

        approachbull Use outer 200 genes

        (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

        (finds significant bumps etc in histogram)

        bull Carefully redrew boundariesbull Check by studying kde angles

        SiZer Study of Distrsquon of Angles

        Reclassification of Major Genes

        Compare to Previous Classifrsquon

        New Subpopulation View

        New Subpopulation View

        NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

        Detailed Look at PCA

        Now Study ldquoFolklorerdquo More Carefully

        bull BackGround

        bull History

        bull Underpinnings

        (Mathematical amp Computational)

        Good Overall Reference Jolliffe (2002)

        PCA Rediscovery ndash Renaming

        Statistics Principal Component Analysis (PCA)

        PCA Rediscovery ndash Renaming

        Statistics Principal Component Analysis (PCA)

        Social Sciences Factor Analysis (PCA is a subset)

        PCA Rediscovery ndash Renaming

        Statistics Principal Component Analysis (PCA)

        Social Sciences Factor Analysis (PCA is a subset)

        Probability Electrical EngKarhunen ndash Loeve expansion

        PCA Rediscovery ndash Renaming

        Statistics Principal Component Analysis (PCA)

        Social Sciences Factor Analysis (PCA is a subset)

        Probability Electrical EngKarhunen ndash Loeve expansion

        Applied MathematicsProper Orthogonal Decomposition (POD)

        PCA Rediscovery ndash Renaming

        Statistics Principal Component Analysis (PCA)

        Social Sciences Factor Analysis (PCA is a subset)

        Probability Electrical EngKarhunen ndash Loeve expansion

        Applied MathematicsProper Orthogonal Decomposition (POD)

        Geo-Sciences Empirical Orthogonal Functions (EOF)

        An Interesting Historical Note

        The 1st () application of PCA to Functional

        Data Analysis

        An Interesting Historical Note

        The 1st () application of PCA to Functional

        Data Analysis

        Rao (1958)

        1st Paper with ldquoCurves as Data Objectsrdquo

        viewpoint

        Detailed Look at PCA

        Three Important (amp Interesting) Viewpoints

        1 Mathematics

        2 Numerics

        3 Statistics

        Goal Study Interrelationships

        Detailed Look at PCA

        Three Important (amp Interesting) Viewpoints

        1 Mathematics

        2 Numerics

        3 Statistics

        1st Review Linear Alg and Multivar Prob

        Review of Linear Algebra

        Vector Space

        bull set of ldquovectorsrdquo

        bull and ldquoscalarsrdquo (coefficients)

        xa

        Review of Linear Algebra

        Vector Space

        bull set of ldquovectorsrdquo

        bull and ldquoscalarsrdquo (coefficients)

        bull ldquoclosedrdquo under ldquolinear combinationrdquo

        ( in space)

        xa

        i

        ii xa

        Review of Linear Algebra

        Vector Space

        bull set of ldquovectorsrdquo

        bull and ldquoscalarsrdquo (coefficients)

        bull ldquoclosedrdquo under ldquolinear combinationrdquo

        ( in space)

        eg

        ldquo dim Euclidrsquon spacerdquo

        xa

        i

        ii xa

        d

        d

        d xx

        x

        x

        x 1

        1

        d

        Review of Linear Algebra (Cont)

        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

        Review of Linear Algebra (Cont)

        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

        Review of Linear Algebra (Cont)

        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

        Note Planes not Through the Origin

        are not Subspaces

        (Do not Contain )00 x

        Review of Linear Algebra (Cont)

        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

        (all Linear Combos of them =

        = Containing Hyperplane

        through Origin)

        Review of Linear Algebra (Cont)

        Basis of Subspace Set of Vectors that

        bull Span ie Everything is a Lin Com of them

        bull are Linearly Indeprsquot ie Lin Com is Unique

        Review of Linear Algebra (Cont)

        Basis of Subspace Set of Vectors that

        bull Span ie Everything is a Lin Com of them

        bull are Linearly Indeprsquot ie Lin Com is Unique

        bull eg ldquoUnit Vector Basisrdquo

        bull Since

        d

        1

        0

        0

        0

        1

        0

        0

        0

        1

        1

        0

        0

        0

        1

        0

        0

        0

        1

        212

        1

        d

        d

        xxx

        x

        x

        x

        Review of Linear Algebra (Cont)

        Basis Matrix of subspace of

        Given a basis

        create matrix of columns

        dnvv 1

        nddnd

        n

        n

        vv

        vv

        vvB

        1

        111

        1

        Review of Linear Algebra (Cont)

        Then linear combo is a matrix multiplicatrsquon

        where

        n

        iii aBva

        1

        na

        a

        a 1

        Review of Linear Algebra (Cont)

        Then linear combo is a matrix multiplicatrsquon

        where

        Note Right Multiplication Gives

        Linear Combination of Column Vectors

        n

        iii aBva

        1

        na

        a

        a 1

        Review of Linear Algebra (Cont)

        Then linear combo is a matrix multiplicatrsquon

        where

        Check sizes

        n

        iii aBva

        1

        na

        a

        a 1

        )1()(1 nndd

        Review of Linear Algebra (Cont)

        Aside on Matrix Multiplication (linear transformatrsquon)

        For matrices

        Define the Matrix Product

        mkk

        m

        aa

        aa

        A

        1

        111

        nmm

        n

        bb

        bb

        B

        1

        111

        m

        iniik

        m

        iiik

        m

        inii

        m

        iii

        baba

        baba

        AB

        1

        11

        11

        111

        Review of Linear Algebra (Cont)

        Aside on Matrix Multiplication (linear transformatrsquon)

        For matrices

        Define the Matrix Product

        (Inner Products of Rows With Columns )

        mkk

        m

        aa

        aa

        A

        1

        111

        nmm

        n

        bb

        bb

        B

        1

        111

        m

        iniik

        m

        iiik

        m

        inii

        m

        iii

        baba

        baba

        AB

        1

        11

        11

        111

        A B

        Review of Linear Algebra (Cont)

        Aside on Matrix Multiplication (linear transformatrsquon)

        For matrices

        Define the Matrix Product

        (Inner Products of Rows With Columns )

        (Composition of Linear Transformations)

        mkk

        m

        aa

        aa

        A

        1

        111

        nmm

        n

        bb

        bb

        B

        1

        111

        m

        iniik

        m

        iiik

        m

        inii

        m

        iii

        baba

        baba

        AB

        1

        11

        11

        111

        A B

        Review of Linear Algebra (Cont)

        Aside on Matrix Multiplication (linear transformatrsquon)

        For matrices

        Define the Matrix Product

        (Inner Products of Rows With Columns )

        (Composition of Linear Transformations)

        Often Useful to Check Sizes

        mkk

        m

        aa

        aa

        A

        1

        111

        nmm

        n

        bb

        bb

        B

        1

        111

        m

        iniik

        m

        iiik

        m

        inii

        m

        iii

        baba

        baba

        AB

        1

        11

        11

        111

        nmmknk

        A B

        Review of Linear Algebra (Cont)

        Aside on Matrix Multiplication (linear transformatrsquon)

        For matrices

        Define the Matrix Product

        (Inner Products of Rows With Columns )

        (Composition of Linear Transformations)

        Often Useful to Check Sizes

        mkk

        m

        aa

        aa

        A

        1

        111

        nmm

        n

        bb

        bb

        B

        1

        111

        m

        iniik

        m

        iiik

        m

        inii

        m

        iii

        baba

        baba

        AB

        1

        11

        11

        111

        nmmknk

        A B

        Review of Linear Algebra (Cont)

        Matrix Trace

        bull For a Square Matrix

        bull Define

        mmm

        m

        aa

        aa

        A

        1

        111

        m

        iiiaAtr

        1)(

        Review of Linear Algebra (Cont)

        Matrix Trace

        bull For a Square Matrix

        bull Define

        bull Trace Commutes with Matrix Multiplication

        mmm

        m

        aa

        aa

        A

        1

        111

        m

        iiiaAtr

        1)(

        BAtrABtr

        Review of Linear Algebra (Cont)

        Dimension of Subspace (a Notion of ldquoSizerdquo)

        bull Number of Elements in a Basis (Unique)

        Review of Linear Algebra (Cont)

        Dimension of Subspace (a Notion of ldquoSizerdquo)

        bull Number of Elements in a Basis (Unique)

        bull (Use Basis Above)

        bull eg dim of a line is 1

        bull eg dim of a plane is 2

        dd dim

        Review of Linear Algebra (Cont)

        Dimension of Subspace (a Notion of ldquoSizerdquo)

        bull Number of Elements in a Basis (Unique)

        bull (Use Basis Above)

        bull eg dim of a line is 1

        bull eg dim of a plane is 2

        bull Dimension is ldquoDegrees of Freedomrdquo

        (in Statistical Uses eg ANOVA)

        dd dim

        Review of Linear Algebra (Cont)

        Norm of a Vector

        bull in d 21

        21

        1

        2 xxxx td

        jj

        Review of Linear Algebra (Cont)

        Norm of a Vector

        bull in

        bull Idea length of the vector

        d 2121

        1

        2 xxxx td

        jj

        Review of Linear Algebra (Cont)

        Norm of a Vector

        bull in

        bull Idea length of the vector

        bull Note strange properties for high

        eg ldquolength of diagonal of unit cuberdquo =

        d 2121

        1

        2 xxxx td

        jj

        d

        d

        Review of Linear Algebra (Cont)

        Norm of a Vector (cont)

        bull Length Normalized Vector

        (has Length 1 thus on Surf of Unit Sphere

        amp is a Direction Vector)

        x

        x

        Review of Linear Algebra (Cont)

        Norm of a Vector (cont)

        bull Length Normalized Vector

        (has Length 1 thus on Surf of Unit Sphere

        amp is a Direction Vector)

        bull Define Distance as

        x

        x

        yxyxyxyxd t

        Review of Linear Algebra (Cont)

        Inner (Dot Scalar) Product

        bull for Vectors and

        yxyxyx td

        jjj

        1

        x y

        Review of Linear Algebra (Cont)

        Inner (Dot Scalar) Product

        bull for Vectors and

        bull Related to Norm via

        yxyxyx td

        jjj

        1

        x y

        21 xxx

        Review of Linear Algebra (Cont)

        Inner (Dot Scalar) Product (cont)

        bull measures ldquoangle between and rdquo as

        yyxx

        yx

        yx

        yxyxangle

        tt

        t

        11 cos

        cos

        x y

        Review of Linear Algebra (Cont)

        Inner (Dot Scalar) Product (cont)

        bull measures ldquoangle between and rdquo as

        bull key to Orthogonality ie Perpendiculrsquoty

        if and only if

        yyxx

        yx

        yx

        yxyxangle

        tt

        t

        11 cos

        cos

        x y

        yx 0 yx

        Review of Linear Algebra (Cont)

        Orthonormal Basis

        bull All Orthogonal to each other

        ie for

        bull All have Length 1

        ie for

        nvv 1

        1 ii vv

        0 ii vv ii

        ni 1

        Review of Linear Algebra (Cont)

        Orthonormal Basis (cont)

        bull Spectral Representation

        where

        nvv 1

        n

        iii vax

        1

        ii vxa

        Review of Linear Algebra (Cont)

        Orthonormal Basis (cont)

        bull Spectral Representation

        where

        (Coefficient is Inner Product Cool Notation)

        nvv 1

        n

        iii vax

        1

        ii vxa

        Review of Linear Algebra (Cont)

        Orthonormal Basis (cont)

        bull Spectral Representation

        where

        Check

        nvv 1

        n

        iii vax

        1

        ii vxa

        iii

        n

        iii

        n

        iiii avvavvavx

        1

        1

        Review of Linear Algebra (Cont)

        Orthonormal Basis (cont)

        bull Spectral Representation

        where

        Check

        bull Matrix Notation where ie

        For the Basis Matrix

        nvv 1

        n

        iii vax

        1

        ii vxa

        iii

        n

        iii

        n

        iiii avvavvavx

        1

        1

        aBx Bxa tt xBa t

        nvvB 1

        Review of Linear Algebra (Cont)

        Orthonormal Basis (cont)

        bull Spectral Representation

        where

        Check

        bull Matrix Notation where ie

        is called transform of

        (eg Fourier or Wavelet)

        nvv 1

        n

        iii vax

        1

        ii vxa

        iii

        n

        iii

        n

        iiii avvavvavx

        1

        1

        aBx Bxa tt xBa t

        xa

        Review of Linear Algebra (Cont)

        Parseval identity for

        in subsp genrsquod by o n basis

        x

        nvv 1

        2

        1

        22

        1

        2 aavxx

        n

        ii

        n

        ii

        Review of Linear Algebra (Cont)

        Parseval identity for

        in subsp genrsquod by o n basis

        bull Pythagorean theorem

        bull ldquoDecomposition of Energyrdquo

        bull ANOVA - sums of squares

        x

        nvv 1

        2

        1

        22

        1

        2 aavxx

        n

        ii

        n

        ii

        Review of Linear Algebra (Cont)

        Parseval identity for

        in subsp genrsquod by o n basis

        bull Pythagorean theorem

        bull ldquoDecomposition of Energyrdquo

        bull ANOVA - sums of squares

        bull Transform has same length as

        ie ldquorotation in rdquo

        x

        nvv 1

        2

        1

        22

        1

        2 aavxx

        n

        ii

        n

        ii

        a xd

        Projection of a Vector onto a Subspace

        bull Idea Member of that is Closest to

        (ie ldquoBest Approxrsquonrdquo)

        Review of Linear Algebra (Cont)x

        xV

        V

        Projection of a Vector onto a Subspace

        bull Idea Member of that is Closest to

        (ie ldquoBest Approxrsquonrdquo)

        bull Find that Solves

        (ldquoLeast Squaresrdquo)

        Review of Linear Algebra (Cont)x

        xV

        V

        VxPV vxVv

        min

        Projection of a Vector onto a Subspace

        bull Idea Member of that is Closest to

        (ie ldquoBest Approxrsquonrdquo)

        bull Find that Solves

        (ldquoLeast Squaresrdquo)

        bull For Inner Product (Hilbert) Space

        Exists and is Unique

        Review of Linear Algebra (Cont)x

        xV

        V

        VxPV vxVv

        min

        xPV

        Projection of a Vector onto a Subspace (cont)

        bull General Solution in for Basis Matrix

        Review of Linear Algebra (Cont)

        d VB

        xBBBBxP tVV

        tVVV

        1

        Projection of a Vector onto a Subspace (cont)

        bull General Solution in for Basis Matrix

        bull So Projrsquon Operator is Matrix Multrsquon

        (thus projection is another linear operation)

        Review of Linear Algebra (Cont)

        d VB

        xBBBBxP tVV

        tVVV

        1

        tVV

        tVVV BBBBP

        1

        Projection of a Vector onto a Subspace (cont)

        bull General Solution in for Basis Matrix

        bull So Projrsquon Operator is Matrix Multrsquon

        (thus projection is another linear operation)

        (note same operation underlies least squares)

        Review of Linear Algebra (Cont)

        d VB

        xBBBBxP tVV

        tVVV

        1

        tVV

        tVVV BBBBP

        1

        Review of Linear Algebra (Cont)

        Projection using Orthonormal Basis

        bull Basis Matrix is Orthonormal

        nnVtV IBB

        10

        01

        1

        111

        1

        1

        nnn

        n

        ntn

        t

        vvvv

        vvvv

        vv

        v

        v

        nvv 1

        Review of Linear Algebra (Cont)

        Projection using Orthonormal Basis

        bull Basis Matrix is Orthonormal

        bull So =

        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

        (Recall Right Multrsquon)

        nnVtV IBB

        10

        01

        1

        111

        1

        1

        nnn

        n

        ntn

        t

        vvvv

        vvvv

        vv

        v

        v

        xBBxP tVVV

        x V

        nvv 1

        Review of Linear Algebra (Cont)

        Projection using Orthonormal Basis (cont)

        bull For Orthogonal Complement

        and

        V

        xPxPx VV 222xPxPx VV

        Review of Linear Algebra (Cont)

        Projection using Orthonormal Basis (cont)

        bull For Orthogonal Complement

        and

        bull Parseval Inequality

        V

        xPxPx VV 222xPxPx VV

        2

        1

        22

        1

        22 aavxxxP

        n

        ii

        n

        iiV

        Review of Linear Algebra (Cont)

        (Real) Unitary Matrices with

        bull Orthonormal Basis Matrix

        (So All of Above Applies)

        ddU IUU t

        Review of Linear Algebra (Cont)

        (Real) Unitary Matrices with

        bull Orthonormal Basis Matrix

        (So All of Above Applies)

        bull Note Transformrsquon is Distance Preserving

        ddU IUU t

        yxdyxyxyxUyUxUdn

        i ii 2

        1

        Review of Linear Algebra (Cont)

        (Real) Unitary Matrices with

        bull Orthonormal Basis Matrix

        (So All of Above Applies)

        bull Note Transformrsquon is Distance Preserving

        bull Lin Trans (Mult by ) is ~ Rotation

        bull But also Includes ldquoMirror Imagesrdquo

        ddU IUU t

        yxdyxyxyxUyUxUdn

        i ii 2

        1

        U

        Review of Linear Algebra (Cont)

        Singular Value Decomposition (SVD)

        For a Matrix

        Find

        ndX

        Review of Linear Algebra (Cont)

        Singular Value Decomposition (SVD)

        For a Matrix

        Find a Diagonal Matrix

        with Entries

        called Singular Values

        ndX

        ndS

        )min(1 ndss

        Review of Linear Algebra (Cont)

        Singular Value Decomposition (SVD)

        For a Matrix

        Find a Diagonal Matrix

        with Entries

        called Singular Values

        And Unitary (Rotation) Matrices

        (recall )

        ndX

        ndS

        )min(1 ndss

        ddU nnV

        IVVUU tt

        Review of Linear Algebra (Cont)

        Singular Value Decomposition (SVD)

        For a Matrix

        Find a Diagonal Matrix

        with Entries

        called Singular Values

        And Unitary (Rotation) Matrices

        (recall )

        So That

        ndX

        ndS

        )min(1 ndss

        ddU nnV

        IVVUU tt tUSVX

        Review of Linear Algebra (Cont)

        Intuition behind Singular Value Decomposition

        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

        vVSUvVSUvX tt

        Review of Linear Algebra (Cont)

        Intuition behind Singular Value Decomposition

        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

        bull First rotate

        X

        vVSUvVSUvX tt

        Review of Linear Algebra (Cont)

        Intuition behind Singular Value Decomposition

        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

        bull First rotate

        bull Second rescale coordinate axes (by )

        X

        vVSUvVSUvX tt

        is

        Review of Linear Algebra (Cont)

        Intuition behind Singular Value Decomposition

        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

        bull First rotate

        bull Second rescale coordinate axes (by )

        bull Third rotate again

        X

        vVSUvVSUvX tt

        is

        Review of Linear Algebra (Cont)

        Intuition behind Singular Value Decomposition

        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

        bull First rotate

        bull Second rescale coordinate axes (by )

        bull Third rotate again

        bull ie have diagonalized the transformation

        X

        vVSUvVSUvX tt

        is

        SVD Compact Representation

        Useful Labeling

        Singular Values in Increasing Order

        Review of Linear Algebra (Cont)

        )min(1 dnss

        SVD Compact Representation

        Useful Labeling

        Singular Values in Increasing Order

        Note singular values = 0 can be omitted

        (Since do ldquo0-Stretchingrdquo)

        Review of Linear Algebra (Cont)

        )min(1 dnss

        Review of Linear Algebra (Cont)

        r

        SVD Compact Representation

        Useful Labeling

        Singular Values in Increasing Order

        Note singular values = 0 can be omitted

        Let = of positive singular values

        )min(1 dnss

        Review of Linear Algebra (Cont)

        r

        SVD Compact Representation

        Useful Labeling

        Singular Values in Increasing Order

        Note singular values = 0 can be omitted

        Let = of positive singular values

        Then

        Where are truncations of

        trnrrrd VSUX

        VSU

        )min(1 dnss

        Review of Linear Algebra (Cont)

        SVD Full Representation

        =

        Graphics Display Assumes

        ndX ddU ndS nn

        tV

        nd

        Review of Linear Algebra (Cont)

        SVD Full Representation

        =

        Full Rank Basis Matrix

        ndX ddU ndS nn

        tV

        Review of Linear Algebra (Cont)

        SVD Full Representation

        =

        Full Rank Basis Matrix

        All 0s in Bottom

        ndX ddU ndS nn

        tV

        Review of Linear Algebra (Cont)

        SVD Reduced Representation

        =

        These Columns Get 0ed Out

        ndX ddU nnS nn

        tV

        nnd 0

        Review of Linear Algebra (Cont)

        SVD Reduced Representation

        =ndX ndU nnS nn

        tV

        Review of Linear Algebra (Cont)

        SVD Reduced Representation

        =

        Also Some of These May be 0

        ndX ndU nnS nn

        tV

        Review of Linear Algebra (Cont)

        SVD Compact Representation

        = ndX rdU

        rrS nrtV

        0

        Review of Linear Algebra (Cont)

        SVD Compact Representation

        =

        These Get 0ed Out

        ndX rdU

        rrS nrtV

        0

        Review of Linear Algebra (Cont)

        SVD Compact Representation

        = ndX rdU

        rrS nrtV

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition

        For a (Symmetric) Square Matrix

        Find

        ddX

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition

        For a (Symmetric) Square Matrix

        Find a Diagonal Matrix

        Called Eigenvalues

        Convenient Ordering

        ddX

        d

        D

        0

        01

        n 1

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition

        For a (Symmetric) Square Matrix

        Find a Diagonal Matrix

        And an Orthonormal Matrix

        (ie )

        ddX

        d

        D

        0

        01

        ddB

        ddtt IBBBB

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition

        For a (Symmetric) Square Matrix

        Find a Diagonal Matrix

        And an Orthonormal Matrix

        (ie )

        So that ie

        ddX

        d

        D

        0

        01

        ddB

        ddtt IBBBB

        DBBX tBDBX

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

        (looks similar)

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

        Complex (uses )

        VU

        1i

        Review of Linear Algebra (Cont)

        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

        VU

        X

        BVU

        1i

        • Participant Presentations
        • PCA to find clusters
        • Statistical Smoothing
        • Kernel Density Estimation
        • Scatterplot Smoothing
        • Statistical Smoothing (2)
        • SiZer Background
        • SiZer Background (2)
        • SiZer Background (3)
        • SiZer Background (4)
        • SiZer Background (5)
        • SiZer Overview
        • PCA to find clusters (2)
        • PCA to find clusters (3)
        • PCA to find clusters (4)
        • PCA to find clusters (5)
        • PCA to find clusters (6)
        • PCA to find clusters (7)
        • Recall Yeast Cell Cycle Data
        • Yeast Cell Cycle Data FDA View
        • Yeast Cell Cycle Data FDA View (2)
        • Yeast Cell Cycle Data FDA View (3)
        • Yeast Cell Cycles Freq 2 Proj
        • Frequency 2 Analysis
        • Frequency 2 Analysis (2)
        • Frequency 2 Analysis (3)
        • Yeast Cell Cycle
        • SiZer Study of Distrsquon of Angles
        • Reclassification of Major Genes
        • Compare to Previous Classifrsquon
        • New Subpopulation View
        • New Subpopulation View (2)
        • Detailed Look at PCA
        • PCA Rediscovery ndash Renaming
        • PCA Rediscovery ndash Renaming (2)
        • PCA Rediscovery ndash Renaming (3)
        • PCA Rediscovery ndash Renaming (4)
        • PCA Rediscovery ndash Renaming (5)
        • An Interesting Historical Note
        • An Interesting Historical Note (2)
        • Detailed Look at PCA (2)
        • Detailed Look at PCA (3)
        • Review of Linear Algebra
        • Review of Linear Algebra (2)
        • Review of Linear Algebra (3)
        • Review of Linear Algebra (Cont)
        • Review of Linear Algebra (Cont) (2)
        • Review of Linear Algebra (Cont) (3)
        • Review of Linear Algebra (Cont) (4)
        • Review of Linear Algebra (Cont) (5)
        • Review of Linear Algebra (Cont) (6)
        • Review of Linear Algebra (Cont) (7)
        • Review of Linear Algebra (Cont) (8)
        • Review of Linear Algebra (Cont) (9)
        • Review of Linear Algebra (Cont) (10)
        • Review of Linear Algebra (Cont) (11)
        • Review of Linear Algebra (Cont) (12)
        • Review of Linear Algebra (Cont) (13)
        • Review of Linear Algebra (Cont) (14)
        • Review of Linear Algebra (Cont) (15)
        • Review of Linear Algebra (Cont) (16)
        • Review of Linear Algebra (Cont) (17)
        • Review of Linear Algebra (Cont) (18)
        • Review of Linear Algebra (Cont) (19)
        • Review of Linear Algebra (Cont) (20)
        • Review of Linear Algebra (Cont) (21)
        • Review of Linear Algebra (Cont) (22)
        • Review of Linear Algebra (Cont) (23)
        • Review of Linear Algebra (Cont) (24)
        • Review of Linear Algebra (Cont) (25)
        • Review of Linear Algebra (Cont) (26)
        • Review of Linear Algebra (Cont) (27)
        • Review of Linear Algebra (Cont) (28)
        • Review of Linear Algebra (Cont) (29)
        • Review of Linear Algebra (Cont) (30)
        • Review of Linear Algebra (Cont) (31)
        • Review of Linear Algebra (Cont) (32)
        • Review of Linear Algebra (Cont) (33)
        • Review of Linear Algebra (Cont) (34)
        • Review of Linear Algebra (Cont) (35)
        • Review of Linear Algebra (Cont) (36)
        • Review of Linear Algebra (Cont) (37)
        • Review of Linear Algebra (Cont) (38)
        • Review of Linear Algebra (Cont) (39)
        • Review of Linear Algebra (Cont) (40)
        • Review of Linear Algebra (Cont) (41)
        • Review of Linear Algebra (Cont) (42)
        • Review of Linear Algebra (Cont) (43)
        • Review of Linear Algebra (Cont) (44)
        • Review of Linear Algebra (Cont) (45)
        • Review of Linear Algebra (Cont) (46)
        • Review of Linear Algebra (Cont) (47)
        • Review of Linear Algebra (Cont) (48)
        • Review of Linear Algebra (Cont) (49)
        • Review of Linear Algebra (Cont) (50)
        • Review of Linear Algebra (Cont) (51)
        • Review of Linear Algebra (Cont) (52)
        • Review of Linear Algebra (Cont) (53)
        • Review of Linear Algebra (Cont) (54)
        • Review of Linear Algebra (Cont) (55)
        • Review of Linear Algebra (Cont) (56)
        • Review of Linear Algebra (Cont) (57)
        • Review of Linear Algebra (Cont) (58)
        • Review of Linear Algebra (Cont) (59)
        • Review of Linear Algebra (Cont) (60)
        • Review of Linear Algebra (Cont) (61)
        • Review of Linear Algebra (Cont) (62)
        • Review of Linear Algebra (Cont) (63)
        • Review of Linear Algebra (Cont) (64)
        • Review of Linear Algebra (Cont) (65)
        • Review of Linear Algebra (Cont) (66)
        • Review of Linear Algebra (Cont) (67)
        • Review of Linear Algebra (Cont) (68)
        • Review of Linear Algebra (Cont) (69)
        • Review of Linear Algebra (Cont) (70)
        • Review of Linear Algebra (Cont) (71)
        • Review of Linear Algebra (Cont) (72)
        • Review of Linear Algebra (Cont) (73)
        • Review of Linear Algebra (Cont) (74)
        • Review of Linear Algebra (Cont) (75)
        • Review of Linear Algebra (Cont) (76)
        • Review of Linear Algebra (Cont) (77)
        • Review of Linear Algebra (Cont) (78)
        • Review of Linear Algebra (Cont) (79)
        • Review of Linear Algebra (Cont) (80)
        • Review of Linear Algebra (Cont) (81)

          Scatterplot SmoothingEg Bralower Fossil Data ndash some

          smooths

          Statistical SmoothingFundamental Question

          For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

          Smoothingrdquo

          Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

          SiZer BackgroundFun Scale Space Views (Incomes Data)

          Surface View

          SiZer BackgroundSiZer analysis of British Incomes data

          >

          SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

          statistically significant

          Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

          wiggles not significant

          Again all are correct

          just different scales

          SiZer BackgroundScale Space and Kernel Choice

          ie Shape of Window

          Compelling Answer Gaussian

          Only ldquoVariation Diminishingrdquo Kernel Shape

          I e Modes decreases with bandwidth h

          Lindebergh (1994)

          Chaudhuri amp Marron (2000)

          SiZer BackgroundRecall

          Hidalgo

          Stamps

          Data

          >

          SiZer Overview

          Would you like to try smoothing amp

          SiZer

          bull Marron Software Website as Before

          bull In ldquoSmoothingrdquo Directory

          ndash kdeSMm

          ndash nprSMm

          ndash sizerSMm

          bull Recall ldquogtgt help sizerSMrdquo for

          usage

          PCA to find clustersReturn to PCA of Mass Flux Data

          PCA to find clustersSiZer analysis of Mass Flux PC1

          PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

          PCA to find clustersSiZer analysis of Mass Flux PC1

          Also in Curvature

          PCA to find clustersSiZer analysis of Mass Flux PC1

          And in Other Comprsquos

          PCA to find clustersSiZer analysis of Mass Flux PC1

          Conclusion

          bull Found 3 significant clusters

          bull Correspond to 3 known ldquocloud typesrdquo

          bull Worth deeper investigation

          Recall Yeast Cell Cycle Data

          bull ldquoGene Expressionrdquo ndash Micro-array data

          bull Data (after major preprocessing) Expression ldquolevelrdquo of

          bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

          10s)bull Interesting statistical issue

          High Dimension Low Sample Size data

          (HDLSS)

          Yeast Cell Cycle Data FDA View

          Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

          Yeast Cell Cycle Data FDA View

          Periodic genes

          Naiumlve

          approach

          Simple PCA

          Yeast Cell Cycle Data FDA Viewbull Central question which genes are

          ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

          ldquovariationrdquobull PCA finds ldquodirections of maximal

          variationrdquobull Often but not always same as

          ldquointeresting directionsrdquobull Here need better approach to study

          periodicities

          Yeast Cell Cycles Freq 2 Proj

          PCA on

          Freq 2

          Periodic

          Component

          Of Data

          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

          Approach from Zhao Marron amp Wells (2004)

          Frequency 2 Analysis

          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

          classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

          notable differencesbull Now try to improve ldquophase classificationrdquo

          Yeast Cell CycleRevisit ldquophase classificationrdquo

          approachbull Use outer 200 genes

          (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

          (finds significant bumps etc in histogram)

          bull Carefully redrew boundariesbull Check by studying kde angles

          SiZer Study of Distrsquon of Angles

          Reclassification of Major Genes

          Compare to Previous Classifrsquon

          New Subpopulation View

          New Subpopulation View

          NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

          Detailed Look at PCA

          Now Study ldquoFolklorerdquo More Carefully

          bull BackGround

          bull History

          bull Underpinnings

          (Mathematical amp Computational)

          Good Overall Reference Jolliffe (2002)

          PCA Rediscovery ndash Renaming

          Statistics Principal Component Analysis (PCA)

          PCA Rediscovery ndash Renaming

          Statistics Principal Component Analysis (PCA)

          Social Sciences Factor Analysis (PCA is a subset)

          PCA Rediscovery ndash Renaming

          Statistics Principal Component Analysis (PCA)

          Social Sciences Factor Analysis (PCA is a subset)

          Probability Electrical EngKarhunen ndash Loeve expansion

          PCA Rediscovery ndash Renaming

          Statistics Principal Component Analysis (PCA)

          Social Sciences Factor Analysis (PCA is a subset)

          Probability Electrical EngKarhunen ndash Loeve expansion

          Applied MathematicsProper Orthogonal Decomposition (POD)

          PCA Rediscovery ndash Renaming

          Statistics Principal Component Analysis (PCA)

          Social Sciences Factor Analysis (PCA is a subset)

          Probability Electrical EngKarhunen ndash Loeve expansion

          Applied MathematicsProper Orthogonal Decomposition (POD)

          Geo-Sciences Empirical Orthogonal Functions (EOF)

          An Interesting Historical Note

          The 1st () application of PCA to Functional

          Data Analysis

          An Interesting Historical Note

          The 1st () application of PCA to Functional

          Data Analysis

          Rao (1958)

          1st Paper with ldquoCurves as Data Objectsrdquo

          viewpoint

          Detailed Look at PCA

          Three Important (amp Interesting) Viewpoints

          1 Mathematics

          2 Numerics

          3 Statistics

          Goal Study Interrelationships

          Detailed Look at PCA

          Three Important (amp Interesting) Viewpoints

          1 Mathematics

          2 Numerics

          3 Statistics

          1st Review Linear Alg and Multivar Prob

          Review of Linear Algebra

          Vector Space

          bull set of ldquovectorsrdquo

          bull and ldquoscalarsrdquo (coefficients)

          xa

          Review of Linear Algebra

          Vector Space

          bull set of ldquovectorsrdquo

          bull and ldquoscalarsrdquo (coefficients)

          bull ldquoclosedrdquo under ldquolinear combinationrdquo

          ( in space)

          xa

          i

          ii xa

          Review of Linear Algebra

          Vector Space

          bull set of ldquovectorsrdquo

          bull and ldquoscalarsrdquo (coefficients)

          bull ldquoclosedrdquo under ldquolinear combinationrdquo

          ( in space)

          eg

          ldquo dim Euclidrsquon spacerdquo

          xa

          i

          ii xa

          d

          d

          d xx

          x

          x

          x 1

          1

          d

          Review of Linear Algebra (Cont)

          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

          Review of Linear Algebra (Cont)

          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

          Review of Linear Algebra (Cont)

          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

          Note Planes not Through the Origin

          are not Subspaces

          (Do not Contain )00 x

          Review of Linear Algebra (Cont)

          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

          (all Linear Combos of them =

          = Containing Hyperplane

          through Origin)

          Review of Linear Algebra (Cont)

          Basis of Subspace Set of Vectors that

          bull Span ie Everything is a Lin Com of them

          bull are Linearly Indeprsquot ie Lin Com is Unique

          Review of Linear Algebra (Cont)

          Basis of Subspace Set of Vectors that

          bull Span ie Everything is a Lin Com of them

          bull are Linearly Indeprsquot ie Lin Com is Unique

          bull eg ldquoUnit Vector Basisrdquo

          bull Since

          d

          1

          0

          0

          0

          1

          0

          0

          0

          1

          1

          0

          0

          0

          1

          0

          0

          0

          1

          212

          1

          d

          d

          xxx

          x

          x

          x

          Review of Linear Algebra (Cont)

          Basis Matrix of subspace of

          Given a basis

          create matrix of columns

          dnvv 1

          nddnd

          n

          n

          vv

          vv

          vvB

          1

          111

          1

          Review of Linear Algebra (Cont)

          Then linear combo is a matrix multiplicatrsquon

          where

          n

          iii aBva

          1

          na

          a

          a 1

          Review of Linear Algebra (Cont)

          Then linear combo is a matrix multiplicatrsquon

          where

          Note Right Multiplication Gives

          Linear Combination of Column Vectors

          n

          iii aBva

          1

          na

          a

          a 1

          Review of Linear Algebra (Cont)

          Then linear combo is a matrix multiplicatrsquon

          where

          Check sizes

          n

          iii aBva

          1

          na

          a

          a 1

          )1()(1 nndd

          Review of Linear Algebra (Cont)

          Aside on Matrix Multiplication (linear transformatrsquon)

          For matrices

          Define the Matrix Product

          mkk

          m

          aa

          aa

          A

          1

          111

          nmm

          n

          bb

          bb

          B

          1

          111

          m

          iniik

          m

          iiik

          m

          inii

          m

          iii

          baba

          baba

          AB

          1

          11

          11

          111

          Review of Linear Algebra (Cont)

          Aside on Matrix Multiplication (linear transformatrsquon)

          For matrices

          Define the Matrix Product

          (Inner Products of Rows With Columns )

          mkk

          m

          aa

          aa

          A

          1

          111

          nmm

          n

          bb

          bb

          B

          1

          111

          m

          iniik

          m

          iiik

          m

          inii

          m

          iii

          baba

          baba

          AB

          1

          11

          11

          111

          A B

          Review of Linear Algebra (Cont)

          Aside on Matrix Multiplication (linear transformatrsquon)

          For matrices

          Define the Matrix Product

          (Inner Products of Rows With Columns )

          (Composition of Linear Transformations)

          mkk

          m

          aa

          aa

          A

          1

          111

          nmm

          n

          bb

          bb

          B

          1

          111

          m

          iniik

          m

          iiik

          m

          inii

          m

          iii

          baba

          baba

          AB

          1

          11

          11

          111

          A B

          Review of Linear Algebra (Cont)

          Aside on Matrix Multiplication (linear transformatrsquon)

          For matrices

          Define the Matrix Product

          (Inner Products of Rows With Columns )

          (Composition of Linear Transformations)

          Often Useful to Check Sizes

          mkk

          m

          aa

          aa

          A

          1

          111

          nmm

          n

          bb

          bb

          B

          1

          111

          m

          iniik

          m

          iiik

          m

          inii

          m

          iii

          baba

          baba

          AB

          1

          11

          11

          111

          nmmknk

          A B

          Review of Linear Algebra (Cont)

          Aside on Matrix Multiplication (linear transformatrsquon)

          For matrices

          Define the Matrix Product

          (Inner Products of Rows With Columns )

          (Composition of Linear Transformations)

          Often Useful to Check Sizes

          mkk

          m

          aa

          aa

          A

          1

          111

          nmm

          n

          bb

          bb

          B

          1

          111

          m

          iniik

          m

          iiik

          m

          inii

          m

          iii

          baba

          baba

          AB

          1

          11

          11

          111

          nmmknk

          A B

          Review of Linear Algebra (Cont)

          Matrix Trace

          bull For a Square Matrix

          bull Define

          mmm

          m

          aa

          aa

          A

          1

          111

          m

          iiiaAtr

          1)(

          Review of Linear Algebra (Cont)

          Matrix Trace

          bull For a Square Matrix

          bull Define

          bull Trace Commutes with Matrix Multiplication

          mmm

          m

          aa

          aa

          A

          1

          111

          m

          iiiaAtr

          1)(

          BAtrABtr

          Review of Linear Algebra (Cont)

          Dimension of Subspace (a Notion of ldquoSizerdquo)

          bull Number of Elements in a Basis (Unique)

          Review of Linear Algebra (Cont)

          Dimension of Subspace (a Notion of ldquoSizerdquo)

          bull Number of Elements in a Basis (Unique)

          bull (Use Basis Above)

          bull eg dim of a line is 1

          bull eg dim of a plane is 2

          dd dim

          Review of Linear Algebra (Cont)

          Dimension of Subspace (a Notion of ldquoSizerdquo)

          bull Number of Elements in a Basis (Unique)

          bull (Use Basis Above)

          bull eg dim of a line is 1

          bull eg dim of a plane is 2

          bull Dimension is ldquoDegrees of Freedomrdquo

          (in Statistical Uses eg ANOVA)

          dd dim

          Review of Linear Algebra (Cont)

          Norm of a Vector

          bull in d 21

          21

          1

          2 xxxx td

          jj

          Review of Linear Algebra (Cont)

          Norm of a Vector

          bull in

          bull Idea length of the vector

          d 2121

          1

          2 xxxx td

          jj

          Review of Linear Algebra (Cont)

          Norm of a Vector

          bull in

          bull Idea length of the vector

          bull Note strange properties for high

          eg ldquolength of diagonal of unit cuberdquo =

          d 2121

          1

          2 xxxx td

          jj

          d

          d

          Review of Linear Algebra (Cont)

          Norm of a Vector (cont)

          bull Length Normalized Vector

          (has Length 1 thus on Surf of Unit Sphere

          amp is a Direction Vector)

          x

          x

          Review of Linear Algebra (Cont)

          Norm of a Vector (cont)

          bull Length Normalized Vector

          (has Length 1 thus on Surf of Unit Sphere

          amp is a Direction Vector)

          bull Define Distance as

          x

          x

          yxyxyxyxd t

          Review of Linear Algebra (Cont)

          Inner (Dot Scalar) Product

          bull for Vectors and

          yxyxyx td

          jjj

          1

          x y

          Review of Linear Algebra (Cont)

          Inner (Dot Scalar) Product

          bull for Vectors and

          bull Related to Norm via

          yxyxyx td

          jjj

          1

          x y

          21 xxx

          Review of Linear Algebra (Cont)

          Inner (Dot Scalar) Product (cont)

          bull measures ldquoangle between and rdquo as

          yyxx

          yx

          yx

          yxyxangle

          tt

          t

          11 cos

          cos

          x y

          Review of Linear Algebra (Cont)

          Inner (Dot Scalar) Product (cont)

          bull measures ldquoangle between and rdquo as

          bull key to Orthogonality ie Perpendiculrsquoty

          if and only if

          yyxx

          yx

          yx

          yxyxangle

          tt

          t

          11 cos

          cos

          x y

          yx 0 yx

          Review of Linear Algebra (Cont)

          Orthonormal Basis

          bull All Orthogonal to each other

          ie for

          bull All have Length 1

          ie for

          nvv 1

          1 ii vv

          0 ii vv ii

          ni 1

          Review of Linear Algebra (Cont)

          Orthonormal Basis (cont)

          bull Spectral Representation

          where

          nvv 1

          n

          iii vax

          1

          ii vxa

          Review of Linear Algebra (Cont)

          Orthonormal Basis (cont)

          bull Spectral Representation

          where

          (Coefficient is Inner Product Cool Notation)

          nvv 1

          n

          iii vax

          1

          ii vxa

          Review of Linear Algebra (Cont)

          Orthonormal Basis (cont)

          bull Spectral Representation

          where

          Check

          nvv 1

          n

          iii vax

          1

          ii vxa

          iii

          n

          iii

          n

          iiii avvavvavx

          1

          1

          Review of Linear Algebra (Cont)

          Orthonormal Basis (cont)

          bull Spectral Representation

          where

          Check

          bull Matrix Notation where ie

          For the Basis Matrix

          nvv 1

          n

          iii vax

          1

          ii vxa

          iii

          n

          iii

          n

          iiii avvavvavx

          1

          1

          aBx Bxa tt xBa t

          nvvB 1

          Review of Linear Algebra (Cont)

          Orthonormal Basis (cont)

          bull Spectral Representation

          where

          Check

          bull Matrix Notation where ie

          is called transform of

          (eg Fourier or Wavelet)

          nvv 1

          n

          iii vax

          1

          ii vxa

          iii

          n

          iii

          n

          iiii avvavvavx

          1

          1

          aBx Bxa tt xBa t

          xa

          Review of Linear Algebra (Cont)

          Parseval identity for

          in subsp genrsquod by o n basis

          x

          nvv 1

          2

          1

          22

          1

          2 aavxx

          n

          ii

          n

          ii

          Review of Linear Algebra (Cont)

          Parseval identity for

          in subsp genrsquod by o n basis

          bull Pythagorean theorem

          bull ldquoDecomposition of Energyrdquo

          bull ANOVA - sums of squares

          x

          nvv 1

          2

          1

          22

          1

          2 aavxx

          n

          ii

          n

          ii

          Review of Linear Algebra (Cont)

          Parseval identity for

          in subsp genrsquod by o n basis

          bull Pythagorean theorem

          bull ldquoDecomposition of Energyrdquo

          bull ANOVA - sums of squares

          bull Transform has same length as

          ie ldquorotation in rdquo

          x

          nvv 1

          2

          1

          22

          1

          2 aavxx

          n

          ii

          n

          ii

          a xd

          Projection of a Vector onto a Subspace

          bull Idea Member of that is Closest to

          (ie ldquoBest Approxrsquonrdquo)

          Review of Linear Algebra (Cont)x

          xV

          V

          Projection of a Vector onto a Subspace

          bull Idea Member of that is Closest to

          (ie ldquoBest Approxrsquonrdquo)

          bull Find that Solves

          (ldquoLeast Squaresrdquo)

          Review of Linear Algebra (Cont)x

          xV

          V

          VxPV vxVv

          min

          Projection of a Vector onto a Subspace

          bull Idea Member of that is Closest to

          (ie ldquoBest Approxrsquonrdquo)

          bull Find that Solves

          (ldquoLeast Squaresrdquo)

          bull For Inner Product (Hilbert) Space

          Exists and is Unique

          Review of Linear Algebra (Cont)x

          xV

          V

          VxPV vxVv

          min

          xPV

          Projection of a Vector onto a Subspace (cont)

          bull General Solution in for Basis Matrix

          Review of Linear Algebra (Cont)

          d VB

          xBBBBxP tVV

          tVVV

          1

          Projection of a Vector onto a Subspace (cont)

          bull General Solution in for Basis Matrix

          bull So Projrsquon Operator is Matrix Multrsquon

          (thus projection is another linear operation)

          Review of Linear Algebra (Cont)

          d VB

          xBBBBxP tVV

          tVVV

          1

          tVV

          tVVV BBBBP

          1

          Projection of a Vector onto a Subspace (cont)

          bull General Solution in for Basis Matrix

          bull So Projrsquon Operator is Matrix Multrsquon

          (thus projection is another linear operation)

          (note same operation underlies least squares)

          Review of Linear Algebra (Cont)

          d VB

          xBBBBxP tVV

          tVVV

          1

          tVV

          tVVV BBBBP

          1

          Review of Linear Algebra (Cont)

          Projection using Orthonormal Basis

          bull Basis Matrix is Orthonormal

          nnVtV IBB

          10

          01

          1

          111

          1

          1

          nnn

          n

          ntn

          t

          vvvv

          vvvv

          vv

          v

          v

          nvv 1

          Review of Linear Algebra (Cont)

          Projection using Orthonormal Basis

          bull Basis Matrix is Orthonormal

          bull So =

          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

          (Recall Right Multrsquon)

          nnVtV IBB

          10

          01

          1

          111

          1

          1

          nnn

          n

          ntn

          t

          vvvv

          vvvv

          vv

          v

          v

          xBBxP tVVV

          x V

          nvv 1

          Review of Linear Algebra (Cont)

          Projection using Orthonormal Basis (cont)

          bull For Orthogonal Complement

          and

          V

          xPxPx VV 222xPxPx VV

          Review of Linear Algebra (Cont)

          Projection using Orthonormal Basis (cont)

          bull For Orthogonal Complement

          and

          bull Parseval Inequality

          V

          xPxPx VV 222xPxPx VV

          2

          1

          22

          1

          22 aavxxxP

          n

          ii

          n

          iiV

          Review of Linear Algebra (Cont)

          (Real) Unitary Matrices with

          bull Orthonormal Basis Matrix

          (So All of Above Applies)

          ddU IUU t

          Review of Linear Algebra (Cont)

          (Real) Unitary Matrices with

          bull Orthonormal Basis Matrix

          (So All of Above Applies)

          bull Note Transformrsquon is Distance Preserving

          ddU IUU t

          yxdyxyxyxUyUxUdn

          i ii 2

          1

          Review of Linear Algebra (Cont)

          (Real) Unitary Matrices with

          bull Orthonormal Basis Matrix

          (So All of Above Applies)

          bull Note Transformrsquon is Distance Preserving

          bull Lin Trans (Mult by ) is ~ Rotation

          bull But also Includes ldquoMirror Imagesrdquo

          ddU IUU t

          yxdyxyxyxUyUxUdn

          i ii 2

          1

          U

          Review of Linear Algebra (Cont)

          Singular Value Decomposition (SVD)

          For a Matrix

          Find

          ndX

          Review of Linear Algebra (Cont)

          Singular Value Decomposition (SVD)

          For a Matrix

          Find a Diagonal Matrix

          with Entries

          called Singular Values

          ndX

          ndS

          )min(1 ndss

          Review of Linear Algebra (Cont)

          Singular Value Decomposition (SVD)

          For a Matrix

          Find a Diagonal Matrix

          with Entries

          called Singular Values

          And Unitary (Rotation) Matrices

          (recall )

          ndX

          ndS

          )min(1 ndss

          ddU nnV

          IVVUU tt

          Review of Linear Algebra (Cont)

          Singular Value Decomposition (SVD)

          For a Matrix

          Find a Diagonal Matrix

          with Entries

          called Singular Values

          And Unitary (Rotation) Matrices

          (recall )

          So That

          ndX

          ndS

          )min(1 ndss

          ddU nnV

          IVVUU tt tUSVX

          Review of Linear Algebra (Cont)

          Intuition behind Singular Value Decomposition

          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

          vVSUvVSUvX tt

          Review of Linear Algebra (Cont)

          Intuition behind Singular Value Decomposition

          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

          bull First rotate

          X

          vVSUvVSUvX tt

          Review of Linear Algebra (Cont)

          Intuition behind Singular Value Decomposition

          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

          bull First rotate

          bull Second rescale coordinate axes (by )

          X

          vVSUvVSUvX tt

          is

          Review of Linear Algebra (Cont)

          Intuition behind Singular Value Decomposition

          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

          bull First rotate

          bull Second rescale coordinate axes (by )

          bull Third rotate again

          X

          vVSUvVSUvX tt

          is

          Review of Linear Algebra (Cont)

          Intuition behind Singular Value Decomposition

          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

          bull First rotate

          bull Second rescale coordinate axes (by )

          bull Third rotate again

          bull ie have diagonalized the transformation

          X

          vVSUvVSUvX tt

          is

          SVD Compact Representation

          Useful Labeling

          Singular Values in Increasing Order

          Review of Linear Algebra (Cont)

          )min(1 dnss

          SVD Compact Representation

          Useful Labeling

          Singular Values in Increasing Order

          Note singular values = 0 can be omitted

          (Since do ldquo0-Stretchingrdquo)

          Review of Linear Algebra (Cont)

          )min(1 dnss

          Review of Linear Algebra (Cont)

          r

          SVD Compact Representation

          Useful Labeling

          Singular Values in Increasing Order

          Note singular values = 0 can be omitted

          Let = of positive singular values

          )min(1 dnss

          Review of Linear Algebra (Cont)

          r

          SVD Compact Representation

          Useful Labeling

          Singular Values in Increasing Order

          Note singular values = 0 can be omitted

          Let = of positive singular values

          Then

          Where are truncations of

          trnrrrd VSUX

          VSU

          )min(1 dnss

          Review of Linear Algebra (Cont)

          SVD Full Representation

          =

          Graphics Display Assumes

          ndX ddU ndS nn

          tV

          nd

          Review of Linear Algebra (Cont)

          SVD Full Representation

          =

          Full Rank Basis Matrix

          ndX ddU ndS nn

          tV

          Review of Linear Algebra (Cont)

          SVD Full Representation

          =

          Full Rank Basis Matrix

          All 0s in Bottom

          ndX ddU ndS nn

          tV

          Review of Linear Algebra (Cont)

          SVD Reduced Representation

          =

          These Columns Get 0ed Out

          ndX ddU nnS nn

          tV

          nnd 0

          Review of Linear Algebra (Cont)

          SVD Reduced Representation

          =ndX ndU nnS nn

          tV

          Review of Linear Algebra (Cont)

          SVD Reduced Representation

          =

          Also Some of These May be 0

          ndX ndU nnS nn

          tV

          Review of Linear Algebra (Cont)

          SVD Compact Representation

          = ndX rdU

          rrS nrtV

          0

          Review of Linear Algebra (Cont)

          SVD Compact Representation

          =

          These Get 0ed Out

          ndX rdU

          rrS nrtV

          0

          Review of Linear Algebra (Cont)

          SVD Compact Representation

          = ndX rdU

          rrS nrtV

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition

          For a (Symmetric) Square Matrix

          Find

          ddX

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition

          For a (Symmetric) Square Matrix

          Find a Diagonal Matrix

          Called Eigenvalues

          Convenient Ordering

          ddX

          d

          D

          0

          01

          n 1

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition

          For a (Symmetric) Square Matrix

          Find a Diagonal Matrix

          And an Orthonormal Matrix

          (ie )

          ddX

          d

          D

          0

          01

          ddB

          ddtt IBBBB

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition

          For a (Symmetric) Square Matrix

          Find a Diagonal Matrix

          And an Orthonormal Matrix

          (ie )

          So that ie

          ddX

          d

          D

          0

          01

          ddB

          ddtt IBBBB

          DBBX tBDBX

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

          (looks similar)

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

          Complex (uses )

          VU

          1i

          Review of Linear Algebra (Cont)

          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

          VU

          X

          BVU

          1i

          • Participant Presentations
          • PCA to find clusters
          • Statistical Smoothing
          • Kernel Density Estimation
          • Scatterplot Smoothing
          • Statistical Smoothing (2)
          • SiZer Background
          • SiZer Background (2)
          • SiZer Background (3)
          • SiZer Background (4)
          • SiZer Background (5)
          • SiZer Overview
          • PCA to find clusters (2)
          • PCA to find clusters (3)
          • PCA to find clusters (4)
          • PCA to find clusters (5)
          • PCA to find clusters (6)
          • PCA to find clusters (7)
          • Recall Yeast Cell Cycle Data
          • Yeast Cell Cycle Data FDA View
          • Yeast Cell Cycle Data FDA View (2)
          • Yeast Cell Cycle Data FDA View (3)
          • Yeast Cell Cycles Freq 2 Proj
          • Frequency 2 Analysis
          • Frequency 2 Analysis (2)
          • Frequency 2 Analysis (3)
          • Yeast Cell Cycle
          • SiZer Study of Distrsquon of Angles
          • Reclassification of Major Genes
          • Compare to Previous Classifrsquon
          • New Subpopulation View
          • New Subpopulation View (2)
          • Detailed Look at PCA
          • PCA Rediscovery ndash Renaming
          • PCA Rediscovery ndash Renaming (2)
          • PCA Rediscovery ndash Renaming (3)
          • PCA Rediscovery ndash Renaming (4)
          • PCA Rediscovery ndash Renaming (5)
          • An Interesting Historical Note
          • An Interesting Historical Note (2)
          • Detailed Look at PCA (2)
          • Detailed Look at PCA (3)
          • Review of Linear Algebra
          • Review of Linear Algebra (2)
          • Review of Linear Algebra (3)
          • Review of Linear Algebra (Cont)
          • Review of Linear Algebra (Cont) (2)
          • Review of Linear Algebra (Cont) (3)
          • Review of Linear Algebra (Cont) (4)
          • Review of Linear Algebra (Cont) (5)
          • Review of Linear Algebra (Cont) (6)
          • Review of Linear Algebra (Cont) (7)
          • Review of Linear Algebra (Cont) (8)
          • Review of Linear Algebra (Cont) (9)
          • Review of Linear Algebra (Cont) (10)
          • Review of Linear Algebra (Cont) (11)
          • Review of Linear Algebra (Cont) (12)
          • Review of Linear Algebra (Cont) (13)
          • Review of Linear Algebra (Cont) (14)
          • Review of Linear Algebra (Cont) (15)
          • Review of Linear Algebra (Cont) (16)
          • Review of Linear Algebra (Cont) (17)
          • Review of Linear Algebra (Cont) (18)
          • Review of Linear Algebra (Cont) (19)
          • Review of Linear Algebra (Cont) (20)
          • Review of Linear Algebra (Cont) (21)
          • Review of Linear Algebra (Cont) (22)
          • Review of Linear Algebra (Cont) (23)
          • Review of Linear Algebra (Cont) (24)
          • Review of Linear Algebra (Cont) (25)
          • Review of Linear Algebra (Cont) (26)
          • Review of Linear Algebra (Cont) (27)
          • Review of Linear Algebra (Cont) (28)
          • Review of Linear Algebra (Cont) (29)
          • Review of Linear Algebra (Cont) (30)
          • Review of Linear Algebra (Cont) (31)
          • Review of Linear Algebra (Cont) (32)
          • Review of Linear Algebra (Cont) (33)
          • Review of Linear Algebra (Cont) (34)
          • Review of Linear Algebra (Cont) (35)
          • Review of Linear Algebra (Cont) (36)
          • Review of Linear Algebra (Cont) (37)
          • Review of Linear Algebra (Cont) (38)
          • Review of Linear Algebra (Cont) (39)
          • Review of Linear Algebra (Cont) (40)
          • Review of Linear Algebra (Cont) (41)
          • Review of Linear Algebra (Cont) (42)
          • Review of Linear Algebra (Cont) (43)
          • Review of Linear Algebra (Cont) (44)
          • Review of Linear Algebra (Cont) (45)
          • Review of Linear Algebra (Cont) (46)
          • Review of Linear Algebra (Cont) (47)
          • Review of Linear Algebra (Cont) (48)
          • Review of Linear Algebra (Cont) (49)
          • Review of Linear Algebra (Cont) (50)
          • Review of Linear Algebra (Cont) (51)
          • Review of Linear Algebra (Cont) (52)
          • Review of Linear Algebra (Cont) (53)
          • Review of Linear Algebra (Cont) (54)
          • Review of Linear Algebra (Cont) (55)
          • Review of Linear Algebra (Cont) (56)
          • Review of Linear Algebra (Cont) (57)
          • Review of Linear Algebra (Cont) (58)
          • Review of Linear Algebra (Cont) (59)
          • Review of Linear Algebra (Cont) (60)
          • Review of Linear Algebra (Cont) (61)
          • Review of Linear Algebra (Cont) (62)
          • Review of Linear Algebra (Cont) (63)
          • Review of Linear Algebra (Cont) (64)
          • Review of Linear Algebra (Cont) (65)
          • Review of Linear Algebra (Cont) (66)
          • Review of Linear Algebra (Cont) (67)
          • Review of Linear Algebra (Cont) (68)
          • Review of Linear Algebra (Cont) (69)
          • Review of Linear Algebra (Cont) (70)
          • Review of Linear Algebra (Cont) (71)
          • Review of Linear Algebra (Cont) (72)
          • Review of Linear Algebra (Cont) (73)
          • Review of Linear Algebra (Cont) (74)
          • Review of Linear Algebra (Cont) (75)
          • Review of Linear Algebra (Cont) (76)
          • Review of Linear Algebra (Cont) (77)
          • Review of Linear Algebra (Cont) (78)
          • Review of Linear Algebra (Cont) (79)
          • Review of Linear Algebra (Cont) (80)
          • Review of Linear Algebra (Cont) (81)

            Statistical SmoothingFundamental Question

            For both ofbull Density Estimation ldquoHistogramsrdquobull Regression ldquoScatterplot

            Smoothingrdquo

            Which bumps are ldquoreally thererdquovs ldquoartifacts of sampling noiserdquo

            SiZer BackgroundFun Scale Space Views (Incomes Data)

            Surface View

            SiZer BackgroundSiZer analysis of British Incomes data

            >

            SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

            statistically significant

            Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

            wiggles not significant

            Again all are correct

            just different scales

            SiZer BackgroundScale Space and Kernel Choice

            ie Shape of Window

            Compelling Answer Gaussian

            Only ldquoVariation Diminishingrdquo Kernel Shape

            I e Modes decreases with bandwidth h

            Lindebergh (1994)

            Chaudhuri amp Marron (2000)

            SiZer BackgroundRecall

            Hidalgo

            Stamps

            Data

            >

            SiZer Overview

            Would you like to try smoothing amp

            SiZer

            bull Marron Software Website as Before

            bull In ldquoSmoothingrdquo Directory

            ndash kdeSMm

            ndash nprSMm

            ndash sizerSMm

            bull Recall ldquogtgt help sizerSMrdquo for

            usage

            PCA to find clustersReturn to PCA of Mass Flux Data

            PCA to find clustersSiZer analysis of Mass Flux PC1

            PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

            PCA to find clustersSiZer analysis of Mass Flux PC1

            Also in Curvature

            PCA to find clustersSiZer analysis of Mass Flux PC1

            And in Other Comprsquos

            PCA to find clustersSiZer analysis of Mass Flux PC1

            Conclusion

            bull Found 3 significant clusters

            bull Correspond to 3 known ldquocloud typesrdquo

            bull Worth deeper investigation

            Recall Yeast Cell Cycle Data

            bull ldquoGene Expressionrdquo ndash Micro-array data

            bull Data (after major preprocessing) Expression ldquolevelrdquo of

            bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

            10s)bull Interesting statistical issue

            High Dimension Low Sample Size data

            (HDLSS)

            Yeast Cell Cycle Data FDA View

            Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

            Yeast Cell Cycle Data FDA View

            Periodic genes

            Naiumlve

            approach

            Simple PCA

            Yeast Cell Cycle Data FDA Viewbull Central question which genes are

            ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

            ldquovariationrdquobull PCA finds ldquodirections of maximal

            variationrdquobull Often but not always same as

            ldquointeresting directionsrdquobull Here need better approach to study

            periodicities

            Yeast Cell Cycles Freq 2 Proj

            PCA on

            Freq 2

            Periodic

            Component

            Of Data

            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

            Approach from Zhao Marron amp Wells (2004)

            Frequency 2 Analysis

            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

            classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

            notable differencesbull Now try to improve ldquophase classificationrdquo

            Yeast Cell CycleRevisit ldquophase classificationrdquo

            approachbull Use outer 200 genes

            (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

            (finds significant bumps etc in histogram)

            bull Carefully redrew boundariesbull Check by studying kde angles

            SiZer Study of Distrsquon of Angles

            Reclassification of Major Genes

            Compare to Previous Classifrsquon

            New Subpopulation View

            New Subpopulation View

            NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

            Detailed Look at PCA

            Now Study ldquoFolklorerdquo More Carefully

            bull BackGround

            bull History

            bull Underpinnings

            (Mathematical amp Computational)

            Good Overall Reference Jolliffe (2002)

            PCA Rediscovery ndash Renaming

            Statistics Principal Component Analysis (PCA)

            PCA Rediscovery ndash Renaming

            Statistics Principal Component Analysis (PCA)

            Social Sciences Factor Analysis (PCA is a subset)

            PCA Rediscovery ndash Renaming

            Statistics Principal Component Analysis (PCA)

            Social Sciences Factor Analysis (PCA is a subset)

            Probability Electrical EngKarhunen ndash Loeve expansion

            PCA Rediscovery ndash Renaming

            Statistics Principal Component Analysis (PCA)

            Social Sciences Factor Analysis (PCA is a subset)

            Probability Electrical EngKarhunen ndash Loeve expansion

            Applied MathematicsProper Orthogonal Decomposition (POD)

            PCA Rediscovery ndash Renaming

            Statistics Principal Component Analysis (PCA)

            Social Sciences Factor Analysis (PCA is a subset)

            Probability Electrical EngKarhunen ndash Loeve expansion

            Applied MathematicsProper Orthogonal Decomposition (POD)

            Geo-Sciences Empirical Orthogonal Functions (EOF)

            An Interesting Historical Note

            The 1st () application of PCA to Functional

            Data Analysis

            An Interesting Historical Note

            The 1st () application of PCA to Functional

            Data Analysis

            Rao (1958)

            1st Paper with ldquoCurves as Data Objectsrdquo

            viewpoint

            Detailed Look at PCA

            Three Important (amp Interesting) Viewpoints

            1 Mathematics

            2 Numerics

            3 Statistics

            Goal Study Interrelationships

            Detailed Look at PCA

            Three Important (amp Interesting) Viewpoints

            1 Mathematics

            2 Numerics

            3 Statistics

            1st Review Linear Alg and Multivar Prob

            Review of Linear Algebra

            Vector Space

            bull set of ldquovectorsrdquo

            bull and ldquoscalarsrdquo (coefficients)

            xa

            Review of Linear Algebra

            Vector Space

            bull set of ldquovectorsrdquo

            bull and ldquoscalarsrdquo (coefficients)

            bull ldquoclosedrdquo under ldquolinear combinationrdquo

            ( in space)

            xa

            i

            ii xa

            Review of Linear Algebra

            Vector Space

            bull set of ldquovectorsrdquo

            bull and ldquoscalarsrdquo (coefficients)

            bull ldquoclosedrdquo under ldquolinear combinationrdquo

            ( in space)

            eg

            ldquo dim Euclidrsquon spacerdquo

            xa

            i

            ii xa

            d

            d

            d xx

            x

            x

            x 1

            1

            d

            Review of Linear Algebra (Cont)

            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

            Review of Linear Algebra (Cont)

            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

            Review of Linear Algebra (Cont)

            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

            Note Planes not Through the Origin

            are not Subspaces

            (Do not Contain )00 x

            Review of Linear Algebra (Cont)

            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

            (all Linear Combos of them =

            = Containing Hyperplane

            through Origin)

            Review of Linear Algebra (Cont)

            Basis of Subspace Set of Vectors that

            bull Span ie Everything is a Lin Com of them

            bull are Linearly Indeprsquot ie Lin Com is Unique

            Review of Linear Algebra (Cont)

            Basis of Subspace Set of Vectors that

            bull Span ie Everything is a Lin Com of them

            bull are Linearly Indeprsquot ie Lin Com is Unique

            bull eg ldquoUnit Vector Basisrdquo

            bull Since

            d

            1

            0

            0

            0

            1

            0

            0

            0

            1

            1

            0

            0

            0

            1

            0

            0

            0

            1

            212

            1

            d

            d

            xxx

            x

            x

            x

            Review of Linear Algebra (Cont)

            Basis Matrix of subspace of

            Given a basis

            create matrix of columns

            dnvv 1

            nddnd

            n

            n

            vv

            vv

            vvB

            1

            111

            1

            Review of Linear Algebra (Cont)

            Then linear combo is a matrix multiplicatrsquon

            where

            n

            iii aBva

            1

            na

            a

            a 1

            Review of Linear Algebra (Cont)

            Then linear combo is a matrix multiplicatrsquon

            where

            Note Right Multiplication Gives

            Linear Combination of Column Vectors

            n

            iii aBva

            1

            na

            a

            a 1

            Review of Linear Algebra (Cont)

            Then linear combo is a matrix multiplicatrsquon

            where

            Check sizes

            n

            iii aBva

            1

            na

            a

            a 1

            )1()(1 nndd

            Review of Linear Algebra (Cont)

            Aside on Matrix Multiplication (linear transformatrsquon)

            For matrices

            Define the Matrix Product

            mkk

            m

            aa

            aa

            A

            1

            111

            nmm

            n

            bb

            bb

            B

            1

            111

            m

            iniik

            m

            iiik

            m

            inii

            m

            iii

            baba

            baba

            AB

            1

            11

            11

            111

            Review of Linear Algebra (Cont)

            Aside on Matrix Multiplication (linear transformatrsquon)

            For matrices

            Define the Matrix Product

            (Inner Products of Rows With Columns )

            mkk

            m

            aa

            aa

            A

            1

            111

            nmm

            n

            bb

            bb

            B

            1

            111

            m

            iniik

            m

            iiik

            m

            inii

            m

            iii

            baba

            baba

            AB

            1

            11

            11

            111

            A B

            Review of Linear Algebra (Cont)

            Aside on Matrix Multiplication (linear transformatrsquon)

            For matrices

            Define the Matrix Product

            (Inner Products of Rows With Columns )

            (Composition of Linear Transformations)

            mkk

            m

            aa

            aa

            A

            1

            111

            nmm

            n

            bb

            bb

            B

            1

            111

            m

            iniik

            m

            iiik

            m

            inii

            m

            iii

            baba

            baba

            AB

            1

            11

            11

            111

            A B

            Review of Linear Algebra (Cont)

            Aside on Matrix Multiplication (linear transformatrsquon)

            For matrices

            Define the Matrix Product

            (Inner Products of Rows With Columns )

            (Composition of Linear Transformations)

            Often Useful to Check Sizes

            mkk

            m

            aa

            aa

            A

            1

            111

            nmm

            n

            bb

            bb

            B

            1

            111

            m

            iniik

            m

            iiik

            m

            inii

            m

            iii

            baba

            baba

            AB

            1

            11

            11

            111

            nmmknk

            A B

            Review of Linear Algebra (Cont)

            Aside on Matrix Multiplication (linear transformatrsquon)

            For matrices

            Define the Matrix Product

            (Inner Products of Rows With Columns )

            (Composition of Linear Transformations)

            Often Useful to Check Sizes

            mkk

            m

            aa

            aa

            A

            1

            111

            nmm

            n

            bb

            bb

            B

            1

            111

            m

            iniik

            m

            iiik

            m

            inii

            m

            iii

            baba

            baba

            AB

            1

            11

            11

            111

            nmmknk

            A B

            Review of Linear Algebra (Cont)

            Matrix Trace

            bull For a Square Matrix

            bull Define

            mmm

            m

            aa

            aa

            A

            1

            111

            m

            iiiaAtr

            1)(

            Review of Linear Algebra (Cont)

            Matrix Trace

            bull For a Square Matrix

            bull Define

            bull Trace Commutes with Matrix Multiplication

            mmm

            m

            aa

            aa

            A

            1

            111

            m

            iiiaAtr

            1)(

            BAtrABtr

            Review of Linear Algebra (Cont)

            Dimension of Subspace (a Notion of ldquoSizerdquo)

            bull Number of Elements in a Basis (Unique)

            Review of Linear Algebra (Cont)

            Dimension of Subspace (a Notion of ldquoSizerdquo)

            bull Number of Elements in a Basis (Unique)

            bull (Use Basis Above)

            bull eg dim of a line is 1

            bull eg dim of a plane is 2

            dd dim

            Review of Linear Algebra (Cont)

            Dimension of Subspace (a Notion of ldquoSizerdquo)

            bull Number of Elements in a Basis (Unique)

            bull (Use Basis Above)

            bull eg dim of a line is 1

            bull eg dim of a plane is 2

            bull Dimension is ldquoDegrees of Freedomrdquo

            (in Statistical Uses eg ANOVA)

            dd dim

            Review of Linear Algebra (Cont)

            Norm of a Vector

            bull in d 21

            21

            1

            2 xxxx td

            jj

            Review of Linear Algebra (Cont)

            Norm of a Vector

            bull in

            bull Idea length of the vector

            d 2121

            1

            2 xxxx td

            jj

            Review of Linear Algebra (Cont)

            Norm of a Vector

            bull in

            bull Idea length of the vector

            bull Note strange properties for high

            eg ldquolength of diagonal of unit cuberdquo =

            d 2121

            1

            2 xxxx td

            jj

            d

            d

            Review of Linear Algebra (Cont)

            Norm of a Vector (cont)

            bull Length Normalized Vector

            (has Length 1 thus on Surf of Unit Sphere

            amp is a Direction Vector)

            x

            x

            Review of Linear Algebra (Cont)

            Norm of a Vector (cont)

            bull Length Normalized Vector

            (has Length 1 thus on Surf of Unit Sphere

            amp is a Direction Vector)

            bull Define Distance as

            x

            x

            yxyxyxyxd t

            Review of Linear Algebra (Cont)

            Inner (Dot Scalar) Product

            bull for Vectors and

            yxyxyx td

            jjj

            1

            x y

            Review of Linear Algebra (Cont)

            Inner (Dot Scalar) Product

            bull for Vectors and

            bull Related to Norm via

            yxyxyx td

            jjj

            1

            x y

            21 xxx

            Review of Linear Algebra (Cont)

            Inner (Dot Scalar) Product (cont)

            bull measures ldquoangle between and rdquo as

            yyxx

            yx

            yx

            yxyxangle

            tt

            t

            11 cos

            cos

            x y

            Review of Linear Algebra (Cont)

            Inner (Dot Scalar) Product (cont)

            bull measures ldquoangle between and rdquo as

            bull key to Orthogonality ie Perpendiculrsquoty

            if and only if

            yyxx

            yx

            yx

            yxyxangle

            tt

            t

            11 cos

            cos

            x y

            yx 0 yx

            Review of Linear Algebra (Cont)

            Orthonormal Basis

            bull All Orthogonal to each other

            ie for

            bull All have Length 1

            ie for

            nvv 1

            1 ii vv

            0 ii vv ii

            ni 1

            Review of Linear Algebra (Cont)

            Orthonormal Basis (cont)

            bull Spectral Representation

            where

            nvv 1

            n

            iii vax

            1

            ii vxa

            Review of Linear Algebra (Cont)

            Orthonormal Basis (cont)

            bull Spectral Representation

            where

            (Coefficient is Inner Product Cool Notation)

            nvv 1

            n

            iii vax

            1

            ii vxa

            Review of Linear Algebra (Cont)

            Orthonormal Basis (cont)

            bull Spectral Representation

            where

            Check

            nvv 1

            n

            iii vax

            1

            ii vxa

            iii

            n

            iii

            n

            iiii avvavvavx

            1

            1

            Review of Linear Algebra (Cont)

            Orthonormal Basis (cont)

            bull Spectral Representation

            where

            Check

            bull Matrix Notation where ie

            For the Basis Matrix

            nvv 1

            n

            iii vax

            1

            ii vxa

            iii

            n

            iii

            n

            iiii avvavvavx

            1

            1

            aBx Bxa tt xBa t

            nvvB 1

            Review of Linear Algebra (Cont)

            Orthonormal Basis (cont)

            bull Spectral Representation

            where

            Check

            bull Matrix Notation where ie

            is called transform of

            (eg Fourier or Wavelet)

            nvv 1

            n

            iii vax

            1

            ii vxa

            iii

            n

            iii

            n

            iiii avvavvavx

            1

            1

            aBx Bxa tt xBa t

            xa

            Review of Linear Algebra (Cont)

            Parseval identity for

            in subsp genrsquod by o n basis

            x

            nvv 1

            2

            1

            22

            1

            2 aavxx

            n

            ii

            n

            ii

            Review of Linear Algebra (Cont)

            Parseval identity for

            in subsp genrsquod by o n basis

            bull Pythagorean theorem

            bull ldquoDecomposition of Energyrdquo

            bull ANOVA - sums of squares

            x

            nvv 1

            2

            1

            22

            1

            2 aavxx

            n

            ii

            n

            ii

            Review of Linear Algebra (Cont)

            Parseval identity for

            in subsp genrsquod by o n basis

            bull Pythagorean theorem

            bull ldquoDecomposition of Energyrdquo

            bull ANOVA - sums of squares

            bull Transform has same length as

            ie ldquorotation in rdquo

            x

            nvv 1

            2

            1

            22

            1

            2 aavxx

            n

            ii

            n

            ii

            a xd

            Projection of a Vector onto a Subspace

            bull Idea Member of that is Closest to

            (ie ldquoBest Approxrsquonrdquo)

            Review of Linear Algebra (Cont)x

            xV

            V

            Projection of a Vector onto a Subspace

            bull Idea Member of that is Closest to

            (ie ldquoBest Approxrsquonrdquo)

            bull Find that Solves

            (ldquoLeast Squaresrdquo)

            Review of Linear Algebra (Cont)x

            xV

            V

            VxPV vxVv

            min

            Projection of a Vector onto a Subspace

            bull Idea Member of that is Closest to

            (ie ldquoBest Approxrsquonrdquo)

            bull Find that Solves

            (ldquoLeast Squaresrdquo)

            bull For Inner Product (Hilbert) Space

            Exists and is Unique

            Review of Linear Algebra (Cont)x

            xV

            V

            VxPV vxVv

            min

            xPV

            Projection of a Vector onto a Subspace (cont)

            bull General Solution in for Basis Matrix

            Review of Linear Algebra (Cont)

            d VB

            xBBBBxP tVV

            tVVV

            1

            Projection of a Vector onto a Subspace (cont)

            bull General Solution in for Basis Matrix

            bull So Projrsquon Operator is Matrix Multrsquon

            (thus projection is another linear operation)

            Review of Linear Algebra (Cont)

            d VB

            xBBBBxP tVV

            tVVV

            1

            tVV

            tVVV BBBBP

            1

            Projection of a Vector onto a Subspace (cont)

            bull General Solution in for Basis Matrix

            bull So Projrsquon Operator is Matrix Multrsquon

            (thus projection is another linear operation)

            (note same operation underlies least squares)

            Review of Linear Algebra (Cont)

            d VB

            xBBBBxP tVV

            tVVV

            1

            tVV

            tVVV BBBBP

            1

            Review of Linear Algebra (Cont)

            Projection using Orthonormal Basis

            bull Basis Matrix is Orthonormal

            nnVtV IBB

            10

            01

            1

            111

            1

            1

            nnn

            n

            ntn

            t

            vvvv

            vvvv

            vv

            v

            v

            nvv 1

            Review of Linear Algebra (Cont)

            Projection using Orthonormal Basis

            bull Basis Matrix is Orthonormal

            bull So =

            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

            (Recall Right Multrsquon)

            nnVtV IBB

            10

            01

            1

            111

            1

            1

            nnn

            n

            ntn

            t

            vvvv

            vvvv

            vv

            v

            v

            xBBxP tVVV

            x V

            nvv 1

            Review of Linear Algebra (Cont)

            Projection using Orthonormal Basis (cont)

            bull For Orthogonal Complement

            and

            V

            xPxPx VV 222xPxPx VV

            Review of Linear Algebra (Cont)

            Projection using Orthonormal Basis (cont)

            bull For Orthogonal Complement

            and

            bull Parseval Inequality

            V

            xPxPx VV 222xPxPx VV

            2

            1

            22

            1

            22 aavxxxP

            n

            ii

            n

            iiV

            Review of Linear Algebra (Cont)

            (Real) Unitary Matrices with

            bull Orthonormal Basis Matrix

            (So All of Above Applies)

            ddU IUU t

            Review of Linear Algebra (Cont)

            (Real) Unitary Matrices with

            bull Orthonormal Basis Matrix

            (So All of Above Applies)

            bull Note Transformrsquon is Distance Preserving

            ddU IUU t

            yxdyxyxyxUyUxUdn

            i ii 2

            1

            Review of Linear Algebra (Cont)

            (Real) Unitary Matrices with

            bull Orthonormal Basis Matrix

            (So All of Above Applies)

            bull Note Transformrsquon is Distance Preserving

            bull Lin Trans (Mult by ) is ~ Rotation

            bull But also Includes ldquoMirror Imagesrdquo

            ddU IUU t

            yxdyxyxyxUyUxUdn

            i ii 2

            1

            U

            Review of Linear Algebra (Cont)

            Singular Value Decomposition (SVD)

            For a Matrix

            Find

            ndX

            Review of Linear Algebra (Cont)

            Singular Value Decomposition (SVD)

            For a Matrix

            Find a Diagonal Matrix

            with Entries

            called Singular Values

            ndX

            ndS

            )min(1 ndss

            Review of Linear Algebra (Cont)

            Singular Value Decomposition (SVD)

            For a Matrix

            Find a Diagonal Matrix

            with Entries

            called Singular Values

            And Unitary (Rotation) Matrices

            (recall )

            ndX

            ndS

            )min(1 ndss

            ddU nnV

            IVVUU tt

            Review of Linear Algebra (Cont)

            Singular Value Decomposition (SVD)

            For a Matrix

            Find a Diagonal Matrix

            with Entries

            called Singular Values

            And Unitary (Rotation) Matrices

            (recall )

            So That

            ndX

            ndS

            )min(1 ndss

            ddU nnV

            IVVUU tt tUSVX

            Review of Linear Algebra (Cont)

            Intuition behind Singular Value Decomposition

            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

            vVSUvVSUvX tt

            Review of Linear Algebra (Cont)

            Intuition behind Singular Value Decomposition

            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

            bull First rotate

            X

            vVSUvVSUvX tt

            Review of Linear Algebra (Cont)

            Intuition behind Singular Value Decomposition

            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

            bull First rotate

            bull Second rescale coordinate axes (by )

            X

            vVSUvVSUvX tt

            is

            Review of Linear Algebra (Cont)

            Intuition behind Singular Value Decomposition

            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

            bull First rotate

            bull Second rescale coordinate axes (by )

            bull Third rotate again

            X

            vVSUvVSUvX tt

            is

            Review of Linear Algebra (Cont)

            Intuition behind Singular Value Decomposition

            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

            bull First rotate

            bull Second rescale coordinate axes (by )

            bull Third rotate again

            bull ie have diagonalized the transformation

            X

            vVSUvVSUvX tt

            is

            SVD Compact Representation

            Useful Labeling

            Singular Values in Increasing Order

            Review of Linear Algebra (Cont)

            )min(1 dnss

            SVD Compact Representation

            Useful Labeling

            Singular Values in Increasing Order

            Note singular values = 0 can be omitted

            (Since do ldquo0-Stretchingrdquo)

            Review of Linear Algebra (Cont)

            )min(1 dnss

            Review of Linear Algebra (Cont)

            r

            SVD Compact Representation

            Useful Labeling

            Singular Values in Increasing Order

            Note singular values = 0 can be omitted

            Let = of positive singular values

            )min(1 dnss

            Review of Linear Algebra (Cont)

            r

            SVD Compact Representation

            Useful Labeling

            Singular Values in Increasing Order

            Note singular values = 0 can be omitted

            Let = of positive singular values

            Then

            Where are truncations of

            trnrrrd VSUX

            VSU

            )min(1 dnss

            Review of Linear Algebra (Cont)

            SVD Full Representation

            =

            Graphics Display Assumes

            ndX ddU ndS nn

            tV

            nd

            Review of Linear Algebra (Cont)

            SVD Full Representation

            =

            Full Rank Basis Matrix

            ndX ddU ndS nn

            tV

            Review of Linear Algebra (Cont)

            SVD Full Representation

            =

            Full Rank Basis Matrix

            All 0s in Bottom

            ndX ddU ndS nn

            tV

            Review of Linear Algebra (Cont)

            SVD Reduced Representation

            =

            These Columns Get 0ed Out

            ndX ddU nnS nn

            tV

            nnd 0

            Review of Linear Algebra (Cont)

            SVD Reduced Representation

            =ndX ndU nnS nn

            tV

            Review of Linear Algebra (Cont)

            SVD Reduced Representation

            =

            Also Some of These May be 0

            ndX ndU nnS nn

            tV

            Review of Linear Algebra (Cont)

            SVD Compact Representation

            = ndX rdU

            rrS nrtV

            0

            Review of Linear Algebra (Cont)

            SVD Compact Representation

            =

            These Get 0ed Out

            ndX rdU

            rrS nrtV

            0

            Review of Linear Algebra (Cont)

            SVD Compact Representation

            = ndX rdU

            rrS nrtV

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition

            For a (Symmetric) Square Matrix

            Find

            ddX

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition

            For a (Symmetric) Square Matrix

            Find a Diagonal Matrix

            Called Eigenvalues

            Convenient Ordering

            ddX

            d

            D

            0

            01

            n 1

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition

            For a (Symmetric) Square Matrix

            Find a Diagonal Matrix

            And an Orthonormal Matrix

            (ie )

            ddX

            d

            D

            0

            01

            ddB

            ddtt IBBBB

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition

            For a (Symmetric) Square Matrix

            Find a Diagonal Matrix

            And an Orthonormal Matrix

            (ie )

            So that ie

            ddX

            d

            D

            0

            01

            ddB

            ddtt IBBBB

            DBBX tBDBX

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

            (looks similar)

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

            Complex (uses )

            VU

            1i

            Review of Linear Algebra (Cont)

            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

            VU

            X

            BVU

            1i

            • Participant Presentations
            • PCA to find clusters
            • Statistical Smoothing
            • Kernel Density Estimation
            • Scatterplot Smoothing
            • Statistical Smoothing (2)
            • SiZer Background
            • SiZer Background (2)
            • SiZer Background (3)
            • SiZer Background (4)
            • SiZer Background (5)
            • SiZer Overview
            • PCA to find clusters (2)
            • PCA to find clusters (3)
            • PCA to find clusters (4)
            • PCA to find clusters (5)
            • PCA to find clusters (6)
            • PCA to find clusters (7)
            • Recall Yeast Cell Cycle Data
            • Yeast Cell Cycle Data FDA View
            • Yeast Cell Cycle Data FDA View (2)
            • Yeast Cell Cycle Data FDA View (3)
            • Yeast Cell Cycles Freq 2 Proj
            • Frequency 2 Analysis
            • Frequency 2 Analysis (2)
            • Frequency 2 Analysis (3)
            • Yeast Cell Cycle
            • SiZer Study of Distrsquon of Angles
            • Reclassification of Major Genes
            • Compare to Previous Classifrsquon
            • New Subpopulation View
            • New Subpopulation View (2)
            • Detailed Look at PCA
            • PCA Rediscovery ndash Renaming
            • PCA Rediscovery ndash Renaming (2)
            • PCA Rediscovery ndash Renaming (3)
            • PCA Rediscovery ndash Renaming (4)
            • PCA Rediscovery ndash Renaming (5)
            • An Interesting Historical Note
            • An Interesting Historical Note (2)
            • Detailed Look at PCA (2)
            • Detailed Look at PCA (3)
            • Review of Linear Algebra
            • Review of Linear Algebra (2)
            • Review of Linear Algebra (3)
            • Review of Linear Algebra (Cont)
            • Review of Linear Algebra (Cont) (2)
            • Review of Linear Algebra (Cont) (3)
            • Review of Linear Algebra (Cont) (4)
            • Review of Linear Algebra (Cont) (5)
            • Review of Linear Algebra (Cont) (6)
            • Review of Linear Algebra (Cont) (7)
            • Review of Linear Algebra (Cont) (8)
            • Review of Linear Algebra (Cont) (9)
            • Review of Linear Algebra (Cont) (10)
            • Review of Linear Algebra (Cont) (11)
            • Review of Linear Algebra (Cont) (12)
            • Review of Linear Algebra (Cont) (13)
            • Review of Linear Algebra (Cont) (14)
            • Review of Linear Algebra (Cont) (15)
            • Review of Linear Algebra (Cont) (16)
            • Review of Linear Algebra (Cont) (17)
            • Review of Linear Algebra (Cont) (18)
            • Review of Linear Algebra (Cont) (19)
            • Review of Linear Algebra (Cont) (20)
            • Review of Linear Algebra (Cont) (21)
            • Review of Linear Algebra (Cont) (22)
            • Review of Linear Algebra (Cont) (23)
            • Review of Linear Algebra (Cont) (24)
            • Review of Linear Algebra (Cont) (25)
            • Review of Linear Algebra (Cont) (26)
            • Review of Linear Algebra (Cont) (27)
            • Review of Linear Algebra (Cont) (28)
            • Review of Linear Algebra (Cont) (29)
            • Review of Linear Algebra (Cont) (30)
            • Review of Linear Algebra (Cont) (31)
            • Review of Linear Algebra (Cont) (32)
            • Review of Linear Algebra (Cont) (33)
            • Review of Linear Algebra (Cont) (34)
            • Review of Linear Algebra (Cont) (35)
            • Review of Linear Algebra (Cont) (36)
            • Review of Linear Algebra (Cont) (37)
            • Review of Linear Algebra (Cont) (38)
            • Review of Linear Algebra (Cont) (39)
            • Review of Linear Algebra (Cont) (40)
            • Review of Linear Algebra (Cont) (41)
            • Review of Linear Algebra (Cont) (42)
            • Review of Linear Algebra (Cont) (43)
            • Review of Linear Algebra (Cont) (44)
            • Review of Linear Algebra (Cont) (45)
            • Review of Linear Algebra (Cont) (46)
            • Review of Linear Algebra (Cont) (47)
            • Review of Linear Algebra (Cont) (48)
            • Review of Linear Algebra (Cont) (49)
            • Review of Linear Algebra (Cont) (50)
            • Review of Linear Algebra (Cont) (51)
            • Review of Linear Algebra (Cont) (52)
            • Review of Linear Algebra (Cont) (53)
            • Review of Linear Algebra (Cont) (54)
            • Review of Linear Algebra (Cont) (55)
            • Review of Linear Algebra (Cont) (56)
            • Review of Linear Algebra (Cont) (57)
            • Review of Linear Algebra (Cont) (58)
            • Review of Linear Algebra (Cont) (59)
            • Review of Linear Algebra (Cont) (60)
            • Review of Linear Algebra (Cont) (61)
            • Review of Linear Algebra (Cont) (62)
            • Review of Linear Algebra (Cont) (63)
            • Review of Linear Algebra (Cont) (64)
            • Review of Linear Algebra (Cont) (65)
            • Review of Linear Algebra (Cont) (66)
            • Review of Linear Algebra (Cont) (67)
            • Review of Linear Algebra (Cont) (68)
            • Review of Linear Algebra (Cont) (69)
            • Review of Linear Algebra (Cont) (70)
            • Review of Linear Algebra (Cont) (71)
            • Review of Linear Algebra (Cont) (72)
            • Review of Linear Algebra (Cont) (73)
            • Review of Linear Algebra (Cont) (74)
            • Review of Linear Algebra (Cont) (75)
            • Review of Linear Algebra (Cont) (76)
            • Review of Linear Algebra (Cont) (77)
            • Review of Linear Algebra (Cont) (78)
            • Review of Linear Algebra (Cont) (79)
            • Review of Linear Algebra (Cont) (80)
            • Review of Linear Algebra (Cont) (81)

              SiZer BackgroundFun Scale Space Views (Incomes Data)

              Surface View

              SiZer BackgroundSiZer analysis of British Incomes data

              >

              SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

              statistically significant

              Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

              wiggles not significant

              Again all are correct

              just different scales

              SiZer BackgroundScale Space and Kernel Choice

              ie Shape of Window

              Compelling Answer Gaussian

              Only ldquoVariation Diminishingrdquo Kernel Shape

              I e Modes decreases with bandwidth h

              Lindebergh (1994)

              Chaudhuri amp Marron (2000)

              SiZer BackgroundRecall

              Hidalgo

              Stamps

              Data

              >

              SiZer Overview

              Would you like to try smoothing amp

              SiZer

              bull Marron Software Website as Before

              bull In ldquoSmoothingrdquo Directory

              ndash kdeSMm

              ndash nprSMm

              ndash sizerSMm

              bull Recall ldquogtgt help sizerSMrdquo for

              usage

              PCA to find clustersReturn to PCA of Mass Flux Data

              PCA to find clustersSiZer analysis of Mass Flux PC1

              PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

              PCA to find clustersSiZer analysis of Mass Flux PC1

              Also in Curvature

              PCA to find clustersSiZer analysis of Mass Flux PC1

              And in Other Comprsquos

              PCA to find clustersSiZer analysis of Mass Flux PC1

              Conclusion

              bull Found 3 significant clusters

              bull Correspond to 3 known ldquocloud typesrdquo

              bull Worth deeper investigation

              Recall Yeast Cell Cycle Data

              bull ldquoGene Expressionrdquo ndash Micro-array data

              bull Data (after major preprocessing) Expression ldquolevelrdquo of

              bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

              10s)bull Interesting statistical issue

              High Dimension Low Sample Size data

              (HDLSS)

              Yeast Cell Cycle Data FDA View

              Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

              Yeast Cell Cycle Data FDA View

              Periodic genes

              Naiumlve

              approach

              Simple PCA

              Yeast Cell Cycle Data FDA Viewbull Central question which genes are

              ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

              ldquovariationrdquobull PCA finds ldquodirections of maximal

              variationrdquobull Often but not always same as

              ldquointeresting directionsrdquobull Here need better approach to study

              periodicities

              Yeast Cell Cycles Freq 2 Proj

              PCA on

              Freq 2

              Periodic

              Component

              Of Data

              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

              Approach from Zhao Marron amp Wells (2004)

              Frequency 2 Analysis

              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

              classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

              notable differencesbull Now try to improve ldquophase classificationrdquo

              Yeast Cell CycleRevisit ldquophase classificationrdquo

              approachbull Use outer 200 genes

              (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

              (finds significant bumps etc in histogram)

              bull Carefully redrew boundariesbull Check by studying kde angles

              SiZer Study of Distrsquon of Angles

              Reclassification of Major Genes

              Compare to Previous Classifrsquon

              New Subpopulation View

              New Subpopulation View

              NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

              Detailed Look at PCA

              Now Study ldquoFolklorerdquo More Carefully

              bull BackGround

              bull History

              bull Underpinnings

              (Mathematical amp Computational)

              Good Overall Reference Jolliffe (2002)

              PCA Rediscovery ndash Renaming

              Statistics Principal Component Analysis (PCA)

              PCA Rediscovery ndash Renaming

              Statistics Principal Component Analysis (PCA)

              Social Sciences Factor Analysis (PCA is a subset)

              PCA Rediscovery ndash Renaming

              Statistics Principal Component Analysis (PCA)

              Social Sciences Factor Analysis (PCA is a subset)

              Probability Electrical EngKarhunen ndash Loeve expansion

              PCA Rediscovery ndash Renaming

              Statistics Principal Component Analysis (PCA)

              Social Sciences Factor Analysis (PCA is a subset)

              Probability Electrical EngKarhunen ndash Loeve expansion

              Applied MathematicsProper Orthogonal Decomposition (POD)

              PCA Rediscovery ndash Renaming

              Statistics Principal Component Analysis (PCA)

              Social Sciences Factor Analysis (PCA is a subset)

              Probability Electrical EngKarhunen ndash Loeve expansion

              Applied MathematicsProper Orthogonal Decomposition (POD)

              Geo-Sciences Empirical Orthogonal Functions (EOF)

              An Interesting Historical Note

              The 1st () application of PCA to Functional

              Data Analysis

              An Interesting Historical Note

              The 1st () application of PCA to Functional

              Data Analysis

              Rao (1958)

              1st Paper with ldquoCurves as Data Objectsrdquo

              viewpoint

              Detailed Look at PCA

              Three Important (amp Interesting) Viewpoints

              1 Mathematics

              2 Numerics

              3 Statistics

              Goal Study Interrelationships

              Detailed Look at PCA

              Three Important (amp Interesting) Viewpoints

              1 Mathematics

              2 Numerics

              3 Statistics

              1st Review Linear Alg and Multivar Prob

              Review of Linear Algebra

              Vector Space

              bull set of ldquovectorsrdquo

              bull and ldquoscalarsrdquo (coefficients)

              xa

              Review of Linear Algebra

              Vector Space

              bull set of ldquovectorsrdquo

              bull and ldquoscalarsrdquo (coefficients)

              bull ldquoclosedrdquo under ldquolinear combinationrdquo

              ( in space)

              xa

              i

              ii xa

              Review of Linear Algebra

              Vector Space

              bull set of ldquovectorsrdquo

              bull and ldquoscalarsrdquo (coefficients)

              bull ldquoclosedrdquo under ldquolinear combinationrdquo

              ( in space)

              eg

              ldquo dim Euclidrsquon spacerdquo

              xa

              i

              ii xa

              d

              d

              d xx

              x

              x

              x 1

              1

              d

              Review of Linear Algebra (Cont)

              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

              Review of Linear Algebra (Cont)

              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

              Review of Linear Algebra (Cont)

              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

              Note Planes not Through the Origin

              are not Subspaces

              (Do not Contain )00 x

              Review of Linear Algebra (Cont)

              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

              (all Linear Combos of them =

              = Containing Hyperplane

              through Origin)

              Review of Linear Algebra (Cont)

              Basis of Subspace Set of Vectors that

              bull Span ie Everything is a Lin Com of them

              bull are Linearly Indeprsquot ie Lin Com is Unique

              Review of Linear Algebra (Cont)

              Basis of Subspace Set of Vectors that

              bull Span ie Everything is a Lin Com of them

              bull are Linearly Indeprsquot ie Lin Com is Unique

              bull eg ldquoUnit Vector Basisrdquo

              bull Since

              d

              1

              0

              0

              0

              1

              0

              0

              0

              1

              1

              0

              0

              0

              1

              0

              0

              0

              1

              212

              1

              d

              d

              xxx

              x

              x

              x

              Review of Linear Algebra (Cont)

              Basis Matrix of subspace of

              Given a basis

              create matrix of columns

              dnvv 1

              nddnd

              n

              n

              vv

              vv

              vvB

              1

              111

              1

              Review of Linear Algebra (Cont)

              Then linear combo is a matrix multiplicatrsquon

              where

              n

              iii aBva

              1

              na

              a

              a 1

              Review of Linear Algebra (Cont)

              Then linear combo is a matrix multiplicatrsquon

              where

              Note Right Multiplication Gives

              Linear Combination of Column Vectors

              n

              iii aBva

              1

              na

              a

              a 1

              Review of Linear Algebra (Cont)

              Then linear combo is a matrix multiplicatrsquon

              where

              Check sizes

              n

              iii aBva

              1

              na

              a

              a 1

              )1()(1 nndd

              Review of Linear Algebra (Cont)

              Aside on Matrix Multiplication (linear transformatrsquon)

              For matrices

              Define the Matrix Product

              mkk

              m

              aa

              aa

              A

              1

              111

              nmm

              n

              bb

              bb

              B

              1

              111

              m

              iniik

              m

              iiik

              m

              inii

              m

              iii

              baba

              baba

              AB

              1

              11

              11

              111

              Review of Linear Algebra (Cont)

              Aside on Matrix Multiplication (linear transformatrsquon)

              For matrices

              Define the Matrix Product

              (Inner Products of Rows With Columns )

              mkk

              m

              aa

              aa

              A

              1

              111

              nmm

              n

              bb

              bb

              B

              1

              111

              m

              iniik

              m

              iiik

              m

              inii

              m

              iii

              baba

              baba

              AB

              1

              11

              11

              111

              A B

              Review of Linear Algebra (Cont)

              Aside on Matrix Multiplication (linear transformatrsquon)

              For matrices

              Define the Matrix Product

              (Inner Products of Rows With Columns )

              (Composition of Linear Transformations)

              mkk

              m

              aa

              aa

              A

              1

              111

              nmm

              n

              bb

              bb

              B

              1

              111

              m

              iniik

              m

              iiik

              m

              inii

              m

              iii

              baba

              baba

              AB

              1

              11

              11

              111

              A B

              Review of Linear Algebra (Cont)

              Aside on Matrix Multiplication (linear transformatrsquon)

              For matrices

              Define the Matrix Product

              (Inner Products of Rows With Columns )

              (Composition of Linear Transformations)

              Often Useful to Check Sizes

              mkk

              m

              aa

              aa

              A

              1

              111

              nmm

              n

              bb

              bb

              B

              1

              111

              m

              iniik

              m

              iiik

              m

              inii

              m

              iii

              baba

              baba

              AB

              1

              11

              11

              111

              nmmknk

              A B

              Review of Linear Algebra (Cont)

              Aside on Matrix Multiplication (linear transformatrsquon)

              For matrices

              Define the Matrix Product

              (Inner Products of Rows With Columns )

              (Composition of Linear Transformations)

              Often Useful to Check Sizes

              mkk

              m

              aa

              aa

              A

              1

              111

              nmm

              n

              bb

              bb

              B

              1

              111

              m

              iniik

              m

              iiik

              m

              inii

              m

              iii

              baba

              baba

              AB

              1

              11

              11

              111

              nmmknk

              A B

              Review of Linear Algebra (Cont)

              Matrix Trace

              bull For a Square Matrix

              bull Define

              mmm

              m

              aa

              aa

              A

              1

              111

              m

              iiiaAtr

              1)(

              Review of Linear Algebra (Cont)

              Matrix Trace

              bull For a Square Matrix

              bull Define

              bull Trace Commutes with Matrix Multiplication

              mmm

              m

              aa

              aa

              A

              1

              111

              m

              iiiaAtr

              1)(

              BAtrABtr

              Review of Linear Algebra (Cont)

              Dimension of Subspace (a Notion of ldquoSizerdquo)

              bull Number of Elements in a Basis (Unique)

              Review of Linear Algebra (Cont)

              Dimension of Subspace (a Notion of ldquoSizerdquo)

              bull Number of Elements in a Basis (Unique)

              bull (Use Basis Above)

              bull eg dim of a line is 1

              bull eg dim of a plane is 2

              dd dim

              Review of Linear Algebra (Cont)

              Dimension of Subspace (a Notion of ldquoSizerdquo)

              bull Number of Elements in a Basis (Unique)

              bull (Use Basis Above)

              bull eg dim of a line is 1

              bull eg dim of a plane is 2

              bull Dimension is ldquoDegrees of Freedomrdquo

              (in Statistical Uses eg ANOVA)

              dd dim

              Review of Linear Algebra (Cont)

              Norm of a Vector

              bull in d 21

              21

              1

              2 xxxx td

              jj

              Review of Linear Algebra (Cont)

              Norm of a Vector

              bull in

              bull Idea length of the vector

              d 2121

              1

              2 xxxx td

              jj

              Review of Linear Algebra (Cont)

              Norm of a Vector

              bull in

              bull Idea length of the vector

              bull Note strange properties for high

              eg ldquolength of diagonal of unit cuberdquo =

              d 2121

              1

              2 xxxx td

              jj

              d

              d

              Review of Linear Algebra (Cont)

              Norm of a Vector (cont)

              bull Length Normalized Vector

              (has Length 1 thus on Surf of Unit Sphere

              amp is a Direction Vector)

              x

              x

              Review of Linear Algebra (Cont)

              Norm of a Vector (cont)

              bull Length Normalized Vector

              (has Length 1 thus on Surf of Unit Sphere

              amp is a Direction Vector)

              bull Define Distance as

              x

              x

              yxyxyxyxd t

              Review of Linear Algebra (Cont)

              Inner (Dot Scalar) Product

              bull for Vectors and

              yxyxyx td

              jjj

              1

              x y

              Review of Linear Algebra (Cont)

              Inner (Dot Scalar) Product

              bull for Vectors and

              bull Related to Norm via

              yxyxyx td

              jjj

              1

              x y

              21 xxx

              Review of Linear Algebra (Cont)

              Inner (Dot Scalar) Product (cont)

              bull measures ldquoangle between and rdquo as

              yyxx

              yx

              yx

              yxyxangle

              tt

              t

              11 cos

              cos

              x y

              Review of Linear Algebra (Cont)

              Inner (Dot Scalar) Product (cont)

              bull measures ldquoangle between and rdquo as

              bull key to Orthogonality ie Perpendiculrsquoty

              if and only if

              yyxx

              yx

              yx

              yxyxangle

              tt

              t

              11 cos

              cos

              x y

              yx 0 yx

              Review of Linear Algebra (Cont)

              Orthonormal Basis

              bull All Orthogonal to each other

              ie for

              bull All have Length 1

              ie for

              nvv 1

              1 ii vv

              0 ii vv ii

              ni 1

              Review of Linear Algebra (Cont)

              Orthonormal Basis (cont)

              bull Spectral Representation

              where

              nvv 1

              n

              iii vax

              1

              ii vxa

              Review of Linear Algebra (Cont)

              Orthonormal Basis (cont)

              bull Spectral Representation

              where

              (Coefficient is Inner Product Cool Notation)

              nvv 1

              n

              iii vax

              1

              ii vxa

              Review of Linear Algebra (Cont)

              Orthonormal Basis (cont)

              bull Spectral Representation

              where

              Check

              nvv 1

              n

              iii vax

              1

              ii vxa

              iii

              n

              iii

              n

              iiii avvavvavx

              1

              1

              Review of Linear Algebra (Cont)

              Orthonormal Basis (cont)

              bull Spectral Representation

              where

              Check

              bull Matrix Notation where ie

              For the Basis Matrix

              nvv 1

              n

              iii vax

              1

              ii vxa

              iii

              n

              iii

              n

              iiii avvavvavx

              1

              1

              aBx Bxa tt xBa t

              nvvB 1

              Review of Linear Algebra (Cont)

              Orthonormal Basis (cont)

              bull Spectral Representation

              where

              Check

              bull Matrix Notation where ie

              is called transform of

              (eg Fourier or Wavelet)

              nvv 1

              n

              iii vax

              1

              ii vxa

              iii

              n

              iii

              n

              iiii avvavvavx

              1

              1

              aBx Bxa tt xBa t

              xa

              Review of Linear Algebra (Cont)

              Parseval identity for

              in subsp genrsquod by o n basis

              x

              nvv 1

              2

              1

              22

              1

              2 aavxx

              n

              ii

              n

              ii

              Review of Linear Algebra (Cont)

              Parseval identity for

              in subsp genrsquod by o n basis

              bull Pythagorean theorem

              bull ldquoDecomposition of Energyrdquo

              bull ANOVA - sums of squares

              x

              nvv 1

              2

              1

              22

              1

              2 aavxx

              n

              ii

              n

              ii

              Review of Linear Algebra (Cont)

              Parseval identity for

              in subsp genrsquod by o n basis

              bull Pythagorean theorem

              bull ldquoDecomposition of Energyrdquo

              bull ANOVA - sums of squares

              bull Transform has same length as

              ie ldquorotation in rdquo

              x

              nvv 1

              2

              1

              22

              1

              2 aavxx

              n

              ii

              n

              ii

              a xd

              Projection of a Vector onto a Subspace

              bull Idea Member of that is Closest to

              (ie ldquoBest Approxrsquonrdquo)

              Review of Linear Algebra (Cont)x

              xV

              V

              Projection of a Vector onto a Subspace

              bull Idea Member of that is Closest to

              (ie ldquoBest Approxrsquonrdquo)

              bull Find that Solves

              (ldquoLeast Squaresrdquo)

              Review of Linear Algebra (Cont)x

              xV

              V

              VxPV vxVv

              min

              Projection of a Vector onto a Subspace

              bull Idea Member of that is Closest to

              (ie ldquoBest Approxrsquonrdquo)

              bull Find that Solves

              (ldquoLeast Squaresrdquo)

              bull For Inner Product (Hilbert) Space

              Exists and is Unique

              Review of Linear Algebra (Cont)x

              xV

              V

              VxPV vxVv

              min

              xPV

              Projection of a Vector onto a Subspace (cont)

              bull General Solution in for Basis Matrix

              Review of Linear Algebra (Cont)

              d VB

              xBBBBxP tVV

              tVVV

              1

              Projection of a Vector onto a Subspace (cont)

              bull General Solution in for Basis Matrix

              bull So Projrsquon Operator is Matrix Multrsquon

              (thus projection is another linear operation)

              Review of Linear Algebra (Cont)

              d VB

              xBBBBxP tVV

              tVVV

              1

              tVV

              tVVV BBBBP

              1

              Projection of a Vector onto a Subspace (cont)

              bull General Solution in for Basis Matrix

              bull So Projrsquon Operator is Matrix Multrsquon

              (thus projection is another linear operation)

              (note same operation underlies least squares)

              Review of Linear Algebra (Cont)

              d VB

              xBBBBxP tVV

              tVVV

              1

              tVV

              tVVV BBBBP

              1

              Review of Linear Algebra (Cont)

              Projection using Orthonormal Basis

              bull Basis Matrix is Orthonormal

              nnVtV IBB

              10

              01

              1

              111

              1

              1

              nnn

              n

              ntn

              t

              vvvv

              vvvv

              vv

              v

              v

              nvv 1

              Review of Linear Algebra (Cont)

              Projection using Orthonormal Basis

              bull Basis Matrix is Orthonormal

              bull So =

              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

              (Recall Right Multrsquon)

              nnVtV IBB

              10

              01

              1

              111

              1

              1

              nnn

              n

              ntn

              t

              vvvv

              vvvv

              vv

              v

              v

              xBBxP tVVV

              x V

              nvv 1

              Review of Linear Algebra (Cont)

              Projection using Orthonormal Basis (cont)

              bull For Orthogonal Complement

              and

              V

              xPxPx VV 222xPxPx VV

              Review of Linear Algebra (Cont)

              Projection using Orthonormal Basis (cont)

              bull For Orthogonal Complement

              and

              bull Parseval Inequality

              V

              xPxPx VV 222xPxPx VV

              2

              1

              22

              1

              22 aavxxxP

              n

              ii

              n

              iiV

              Review of Linear Algebra (Cont)

              (Real) Unitary Matrices with

              bull Orthonormal Basis Matrix

              (So All of Above Applies)

              ddU IUU t

              Review of Linear Algebra (Cont)

              (Real) Unitary Matrices with

              bull Orthonormal Basis Matrix

              (So All of Above Applies)

              bull Note Transformrsquon is Distance Preserving

              ddU IUU t

              yxdyxyxyxUyUxUdn

              i ii 2

              1

              Review of Linear Algebra (Cont)

              (Real) Unitary Matrices with

              bull Orthonormal Basis Matrix

              (So All of Above Applies)

              bull Note Transformrsquon is Distance Preserving

              bull Lin Trans (Mult by ) is ~ Rotation

              bull But also Includes ldquoMirror Imagesrdquo

              ddU IUU t

              yxdyxyxyxUyUxUdn

              i ii 2

              1

              U

              Review of Linear Algebra (Cont)

              Singular Value Decomposition (SVD)

              For a Matrix

              Find

              ndX

              Review of Linear Algebra (Cont)

              Singular Value Decomposition (SVD)

              For a Matrix

              Find a Diagonal Matrix

              with Entries

              called Singular Values

              ndX

              ndS

              )min(1 ndss

              Review of Linear Algebra (Cont)

              Singular Value Decomposition (SVD)

              For a Matrix

              Find a Diagonal Matrix

              with Entries

              called Singular Values

              And Unitary (Rotation) Matrices

              (recall )

              ndX

              ndS

              )min(1 ndss

              ddU nnV

              IVVUU tt

              Review of Linear Algebra (Cont)

              Singular Value Decomposition (SVD)

              For a Matrix

              Find a Diagonal Matrix

              with Entries

              called Singular Values

              And Unitary (Rotation) Matrices

              (recall )

              So That

              ndX

              ndS

              )min(1 ndss

              ddU nnV

              IVVUU tt tUSVX

              Review of Linear Algebra (Cont)

              Intuition behind Singular Value Decomposition

              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

              vVSUvVSUvX tt

              Review of Linear Algebra (Cont)

              Intuition behind Singular Value Decomposition

              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

              bull First rotate

              X

              vVSUvVSUvX tt

              Review of Linear Algebra (Cont)

              Intuition behind Singular Value Decomposition

              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

              bull First rotate

              bull Second rescale coordinate axes (by )

              X

              vVSUvVSUvX tt

              is

              Review of Linear Algebra (Cont)

              Intuition behind Singular Value Decomposition

              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

              bull First rotate

              bull Second rescale coordinate axes (by )

              bull Third rotate again

              X

              vVSUvVSUvX tt

              is

              Review of Linear Algebra (Cont)

              Intuition behind Singular Value Decomposition

              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

              bull First rotate

              bull Second rescale coordinate axes (by )

              bull Third rotate again

              bull ie have diagonalized the transformation

              X

              vVSUvVSUvX tt

              is

              SVD Compact Representation

              Useful Labeling

              Singular Values in Increasing Order

              Review of Linear Algebra (Cont)

              )min(1 dnss

              SVD Compact Representation

              Useful Labeling

              Singular Values in Increasing Order

              Note singular values = 0 can be omitted

              (Since do ldquo0-Stretchingrdquo)

              Review of Linear Algebra (Cont)

              )min(1 dnss

              Review of Linear Algebra (Cont)

              r

              SVD Compact Representation

              Useful Labeling

              Singular Values in Increasing Order

              Note singular values = 0 can be omitted

              Let = of positive singular values

              )min(1 dnss

              Review of Linear Algebra (Cont)

              r

              SVD Compact Representation

              Useful Labeling

              Singular Values in Increasing Order

              Note singular values = 0 can be omitted

              Let = of positive singular values

              Then

              Where are truncations of

              trnrrrd VSUX

              VSU

              )min(1 dnss

              Review of Linear Algebra (Cont)

              SVD Full Representation

              =

              Graphics Display Assumes

              ndX ddU ndS nn

              tV

              nd

              Review of Linear Algebra (Cont)

              SVD Full Representation

              =

              Full Rank Basis Matrix

              ndX ddU ndS nn

              tV

              Review of Linear Algebra (Cont)

              SVD Full Representation

              =

              Full Rank Basis Matrix

              All 0s in Bottom

              ndX ddU ndS nn

              tV

              Review of Linear Algebra (Cont)

              SVD Reduced Representation

              =

              These Columns Get 0ed Out

              ndX ddU nnS nn

              tV

              nnd 0

              Review of Linear Algebra (Cont)

              SVD Reduced Representation

              =ndX ndU nnS nn

              tV

              Review of Linear Algebra (Cont)

              SVD Reduced Representation

              =

              Also Some of These May be 0

              ndX ndU nnS nn

              tV

              Review of Linear Algebra (Cont)

              SVD Compact Representation

              = ndX rdU

              rrS nrtV

              0

              Review of Linear Algebra (Cont)

              SVD Compact Representation

              =

              These Get 0ed Out

              ndX rdU

              rrS nrtV

              0

              Review of Linear Algebra (Cont)

              SVD Compact Representation

              = ndX rdU

              rrS nrtV

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition

              For a (Symmetric) Square Matrix

              Find

              ddX

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition

              For a (Symmetric) Square Matrix

              Find a Diagonal Matrix

              Called Eigenvalues

              Convenient Ordering

              ddX

              d

              D

              0

              01

              n 1

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition

              For a (Symmetric) Square Matrix

              Find a Diagonal Matrix

              And an Orthonormal Matrix

              (ie )

              ddX

              d

              D

              0

              01

              ddB

              ddtt IBBBB

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition

              For a (Symmetric) Square Matrix

              Find a Diagonal Matrix

              And an Orthonormal Matrix

              (ie )

              So that ie

              ddX

              d

              D

              0

              01

              ddB

              ddtt IBBBB

              DBBX tBDBX

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

              (looks similar)

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

              Complex (uses )

              VU

              1i

              Review of Linear Algebra (Cont)

              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

              VU

              X

              BVU

              1i

              • Participant Presentations
              • PCA to find clusters
              • Statistical Smoothing
              • Kernel Density Estimation
              • Scatterplot Smoothing
              • Statistical Smoothing (2)
              • SiZer Background
              • SiZer Background (2)
              • SiZer Background (3)
              • SiZer Background (4)
              • SiZer Background (5)
              • SiZer Overview
              • PCA to find clusters (2)
              • PCA to find clusters (3)
              • PCA to find clusters (4)
              • PCA to find clusters (5)
              • PCA to find clusters (6)
              • PCA to find clusters (7)
              • Recall Yeast Cell Cycle Data
              • Yeast Cell Cycle Data FDA View
              • Yeast Cell Cycle Data FDA View (2)
              • Yeast Cell Cycle Data FDA View (3)
              • Yeast Cell Cycles Freq 2 Proj
              • Frequency 2 Analysis
              • Frequency 2 Analysis (2)
              • Frequency 2 Analysis (3)
              • Yeast Cell Cycle
              • SiZer Study of Distrsquon of Angles
              • Reclassification of Major Genes
              • Compare to Previous Classifrsquon
              • New Subpopulation View
              • New Subpopulation View (2)
              • Detailed Look at PCA
              • PCA Rediscovery ndash Renaming
              • PCA Rediscovery ndash Renaming (2)
              • PCA Rediscovery ndash Renaming (3)
              • PCA Rediscovery ndash Renaming (4)
              • PCA Rediscovery ndash Renaming (5)
              • An Interesting Historical Note
              • An Interesting Historical Note (2)
              • Detailed Look at PCA (2)
              • Detailed Look at PCA (3)
              • Review of Linear Algebra
              • Review of Linear Algebra (2)
              • Review of Linear Algebra (3)
              • Review of Linear Algebra (Cont)
              • Review of Linear Algebra (Cont) (2)
              • Review of Linear Algebra (Cont) (3)
              • Review of Linear Algebra (Cont) (4)
              • Review of Linear Algebra (Cont) (5)
              • Review of Linear Algebra (Cont) (6)
              • Review of Linear Algebra (Cont) (7)
              • Review of Linear Algebra (Cont) (8)
              • Review of Linear Algebra (Cont) (9)
              • Review of Linear Algebra (Cont) (10)
              • Review of Linear Algebra (Cont) (11)
              • Review of Linear Algebra (Cont) (12)
              • Review of Linear Algebra (Cont) (13)
              • Review of Linear Algebra (Cont) (14)
              • Review of Linear Algebra (Cont) (15)
              • Review of Linear Algebra (Cont) (16)
              • Review of Linear Algebra (Cont) (17)
              • Review of Linear Algebra (Cont) (18)
              • Review of Linear Algebra (Cont) (19)
              • Review of Linear Algebra (Cont) (20)
              • Review of Linear Algebra (Cont) (21)
              • Review of Linear Algebra (Cont) (22)
              • Review of Linear Algebra (Cont) (23)
              • Review of Linear Algebra (Cont) (24)
              • Review of Linear Algebra (Cont) (25)
              • Review of Linear Algebra (Cont) (26)
              • Review of Linear Algebra (Cont) (27)
              • Review of Linear Algebra (Cont) (28)
              • Review of Linear Algebra (Cont) (29)
              • Review of Linear Algebra (Cont) (30)
              • Review of Linear Algebra (Cont) (31)
              • Review of Linear Algebra (Cont) (32)
              • Review of Linear Algebra (Cont) (33)
              • Review of Linear Algebra (Cont) (34)
              • Review of Linear Algebra (Cont) (35)
              • Review of Linear Algebra (Cont) (36)
              • Review of Linear Algebra (Cont) (37)
              • Review of Linear Algebra (Cont) (38)
              • Review of Linear Algebra (Cont) (39)
              • Review of Linear Algebra (Cont) (40)
              • Review of Linear Algebra (Cont) (41)
              • Review of Linear Algebra (Cont) (42)
              • Review of Linear Algebra (Cont) (43)
              • Review of Linear Algebra (Cont) (44)
              • Review of Linear Algebra (Cont) (45)
              • Review of Linear Algebra (Cont) (46)
              • Review of Linear Algebra (Cont) (47)
              • Review of Linear Algebra (Cont) (48)
              • Review of Linear Algebra (Cont) (49)
              • Review of Linear Algebra (Cont) (50)
              • Review of Linear Algebra (Cont) (51)
              • Review of Linear Algebra (Cont) (52)
              • Review of Linear Algebra (Cont) (53)
              • Review of Linear Algebra (Cont) (54)
              • Review of Linear Algebra (Cont) (55)
              • Review of Linear Algebra (Cont) (56)
              • Review of Linear Algebra (Cont) (57)
              • Review of Linear Algebra (Cont) (58)
              • Review of Linear Algebra (Cont) (59)
              • Review of Linear Algebra (Cont) (60)
              • Review of Linear Algebra (Cont) (61)
              • Review of Linear Algebra (Cont) (62)
              • Review of Linear Algebra (Cont) (63)
              • Review of Linear Algebra (Cont) (64)
              • Review of Linear Algebra (Cont) (65)
              • Review of Linear Algebra (Cont) (66)
              • Review of Linear Algebra (Cont) (67)
              • Review of Linear Algebra (Cont) (68)
              • Review of Linear Algebra (Cont) (69)
              • Review of Linear Algebra (Cont) (70)
              • Review of Linear Algebra (Cont) (71)
              • Review of Linear Algebra (Cont) (72)
              • Review of Linear Algebra (Cont) (73)
              • Review of Linear Algebra (Cont) (74)
              • Review of Linear Algebra (Cont) (75)
              • Review of Linear Algebra (Cont) (76)
              • Review of Linear Algebra (Cont) (77)
              • Review of Linear Algebra (Cont) (78)
              • Review of Linear Algebra (Cont) (79)
              • Review of Linear Algebra (Cont) (80)
              • Review of Linear Algebra (Cont) (81)

                SiZer BackgroundSiZer analysis of British Incomes data

                >

                SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

                statistically significant

                Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

                wiggles not significant

                Again all are correct

                just different scales

                SiZer BackgroundScale Space and Kernel Choice

                ie Shape of Window

                Compelling Answer Gaussian

                Only ldquoVariation Diminishingrdquo Kernel Shape

                I e Modes decreases with bandwidth h

                Lindebergh (1994)

                Chaudhuri amp Marron (2000)

                SiZer BackgroundRecall

                Hidalgo

                Stamps

                Data

                >

                SiZer Overview

                Would you like to try smoothing amp

                SiZer

                bull Marron Software Website as Before

                bull In ldquoSmoothingrdquo Directory

                ndash kdeSMm

                ndash nprSMm

                ndash sizerSMm

                bull Recall ldquogtgt help sizerSMrdquo for

                usage

                PCA to find clustersReturn to PCA of Mass Flux Data

                PCA to find clustersSiZer analysis of Mass Flux PC1

                PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                PCA to find clustersSiZer analysis of Mass Flux PC1

                Also in Curvature

                PCA to find clustersSiZer analysis of Mass Flux PC1

                And in Other Comprsquos

                PCA to find clustersSiZer analysis of Mass Flux PC1

                Conclusion

                bull Found 3 significant clusters

                bull Correspond to 3 known ldquocloud typesrdquo

                bull Worth deeper investigation

                Recall Yeast Cell Cycle Data

                bull ldquoGene Expressionrdquo ndash Micro-array data

                bull Data (after major preprocessing) Expression ldquolevelrdquo of

                bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                10s)bull Interesting statistical issue

                High Dimension Low Sample Size data

                (HDLSS)

                Yeast Cell Cycle Data FDA View

                Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                Yeast Cell Cycle Data FDA View

                Periodic genes

                Naiumlve

                approach

                Simple PCA

                Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                ldquovariationrdquobull PCA finds ldquodirections of maximal

                variationrdquobull Often but not always same as

                ldquointeresting directionsrdquobull Here need better approach to study

                periodicities

                Yeast Cell Cycles Freq 2 Proj

                PCA on

                Freq 2

                Periodic

                Component

                Of Data

                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                Approach from Zhao Marron amp Wells (2004)

                Frequency 2 Analysis

                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                notable differencesbull Now try to improve ldquophase classificationrdquo

                Yeast Cell CycleRevisit ldquophase classificationrdquo

                approachbull Use outer 200 genes

                (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                (finds significant bumps etc in histogram)

                bull Carefully redrew boundariesbull Check by studying kde angles

                SiZer Study of Distrsquon of Angles

                Reclassification of Major Genes

                Compare to Previous Classifrsquon

                New Subpopulation View

                New Subpopulation View

                NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                Detailed Look at PCA

                Now Study ldquoFolklorerdquo More Carefully

                bull BackGround

                bull History

                bull Underpinnings

                (Mathematical amp Computational)

                Good Overall Reference Jolliffe (2002)

                PCA Rediscovery ndash Renaming

                Statistics Principal Component Analysis (PCA)

                PCA Rediscovery ndash Renaming

                Statistics Principal Component Analysis (PCA)

                Social Sciences Factor Analysis (PCA is a subset)

                PCA Rediscovery ndash Renaming

                Statistics Principal Component Analysis (PCA)

                Social Sciences Factor Analysis (PCA is a subset)

                Probability Electrical EngKarhunen ndash Loeve expansion

                PCA Rediscovery ndash Renaming

                Statistics Principal Component Analysis (PCA)

                Social Sciences Factor Analysis (PCA is a subset)

                Probability Electrical EngKarhunen ndash Loeve expansion

                Applied MathematicsProper Orthogonal Decomposition (POD)

                PCA Rediscovery ndash Renaming

                Statistics Principal Component Analysis (PCA)

                Social Sciences Factor Analysis (PCA is a subset)

                Probability Electrical EngKarhunen ndash Loeve expansion

                Applied MathematicsProper Orthogonal Decomposition (POD)

                Geo-Sciences Empirical Orthogonal Functions (EOF)

                An Interesting Historical Note

                The 1st () application of PCA to Functional

                Data Analysis

                An Interesting Historical Note

                The 1st () application of PCA to Functional

                Data Analysis

                Rao (1958)

                1st Paper with ldquoCurves as Data Objectsrdquo

                viewpoint

                Detailed Look at PCA

                Three Important (amp Interesting) Viewpoints

                1 Mathematics

                2 Numerics

                3 Statistics

                Goal Study Interrelationships

                Detailed Look at PCA

                Three Important (amp Interesting) Viewpoints

                1 Mathematics

                2 Numerics

                3 Statistics

                1st Review Linear Alg and Multivar Prob

                Review of Linear Algebra

                Vector Space

                bull set of ldquovectorsrdquo

                bull and ldquoscalarsrdquo (coefficients)

                xa

                Review of Linear Algebra

                Vector Space

                bull set of ldquovectorsrdquo

                bull and ldquoscalarsrdquo (coefficients)

                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                ( in space)

                xa

                i

                ii xa

                Review of Linear Algebra

                Vector Space

                bull set of ldquovectorsrdquo

                bull and ldquoscalarsrdquo (coefficients)

                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                ( in space)

                eg

                ldquo dim Euclidrsquon spacerdquo

                xa

                i

                ii xa

                d

                d

                d xx

                x

                x

                x 1

                1

                d

                Review of Linear Algebra (Cont)

                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                Review of Linear Algebra (Cont)

                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                Review of Linear Algebra (Cont)

                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                Note Planes not Through the Origin

                are not Subspaces

                (Do not Contain )00 x

                Review of Linear Algebra (Cont)

                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                (all Linear Combos of them =

                = Containing Hyperplane

                through Origin)

                Review of Linear Algebra (Cont)

                Basis of Subspace Set of Vectors that

                bull Span ie Everything is a Lin Com of them

                bull are Linearly Indeprsquot ie Lin Com is Unique

                Review of Linear Algebra (Cont)

                Basis of Subspace Set of Vectors that

                bull Span ie Everything is a Lin Com of them

                bull are Linearly Indeprsquot ie Lin Com is Unique

                bull eg ldquoUnit Vector Basisrdquo

                bull Since

                d

                1

                0

                0

                0

                1

                0

                0

                0

                1

                1

                0

                0

                0

                1

                0

                0

                0

                1

                212

                1

                d

                d

                xxx

                x

                x

                x

                Review of Linear Algebra (Cont)

                Basis Matrix of subspace of

                Given a basis

                create matrix of columns

                dnvv 1

                nddnd

                n

                n

                vv

                vv

                vvB

                1

                111

                1

                Review of Linear Algebra (Cont)

                Then linear combo is a matrix multiplicatrsquon

                where

                n

                iii aBva

                1

                na

                a

                a 1

                Review of Linear Algebra (Cont)

                Then linear combo is a matrix multiplicatrsquon

                where

                Note Right Multiplication Gives

                Linear Combination of Column Vectors

                n

                iii aBva

                1

                na

                a

                a 1

                Review of Linear Algebra (Cont)

                Then linear combo is a matrix multiplicatrsquon

                where

                Check sizes

                n

                iii aBva

                1

                na

                a

                a 1

                )1()(1 nndd

                Review of Linear Algebra (Cont)

                Aside on Matrix Multiplication (linear transformatrsquon)

                For matrices

                Define the Matrix Product

                mkk

                m

                aa

                aa

                A

                1

                111

                nmm

                n

                bb

                bb

                B

                1

                111

                m

                iniik

                m

                iiik

                m

                inii

                m

                iii

                baba

                baba

                AB

                1

                11

                11

                111

                Review of Linear Algebra (Cont)

                Aside on Matrix Multiplication (linear transformatrsquon)

                For matrices

                Define the Matrix Product

                (Inner Products of Rows With Columns )

                mkk

                m

                aa

                aa

                A

                1

                111

                nmm

                n

                bb

                bb

                B

                1

                111

                m

                iniik

                m

                iiik

                m

                inii

                m

                iii

                baba

                baba

                AB

                1

                11

                11

                111

                A B

                Review of Linear Algebra (Cont)

                Aside on Matrix Multiplication (linear transformatrsquon)

                For matrices

                Define the Matrix Product

                (Inner Products of Rows With Columns )

                (Composition of Linear Transformations)

                mkk

                m

                aa

                aa

                A

                1

                111

                nmm

                n

                bb

                bb

                B

                1

                111

                m

                iniik

                m

                iiik

                m

                inii

                m

                iii

                baba

                baba

                AB

                1

                11

                11

                111

                A B

                Review of Linear Algebra (Cont)

                Aside on Matrix Multiplication (linear transformatrsquon)

                For matrices

                Define the Matrix Product

                (Inner Products of Rows With Columns )

                (Composition of Linear Transformations)

                Often Useful to Check Sizes

                mkk

                m

                aa

                aa

                A

                1

                111

                nmm

                n

                bb

                bb

                B

                1

                111

                m

                iniik

                m

                iiik

                m

                inii

                m

                iii

                baba

                baba

                AB

                1

                11

                11

                111

                nmmknk

                A B

                Review of Linear Algebra (Cont)

                Aside on Matrix Multiplication (linear transformatrsquon)

                For matrices

                Define the Matrix Product

                (Inner Products of Rows With Columns )

                (Composition of Linear Transformations)

                Often Useful to Check Sizes

                mkk

                m

                aa

                aa

                A

                1

                111

                nmm

                n

                bb

                bb

                B

                1

                111

                m

                iniik

                m

                iiik

                m

                inii

                m

                iii

                baba

                baba

                AB

                1

                11

                11

                111

                nmmknk

                A B

                Review of Linear Algebra (Cont)

                Matrix Trace

                bull For a Square Matrix

                bull Define

                mmm

                m

                aa

                aa

                A

                1

                111

                m

                iiiaAtr

                1)(

                Review of Linear Algebra (Cont)

                Matrix Trace

                bull For a Square Matrix

                bull Define

                bull Trace Commutes with Matrix Multiplication

                mmm

                m

                aa

                aa

                A

                1

                111

                m

                iiiaAtr

                1)(

                BAtrABtr

                Review of Linear Algebra (Cont)

                Dimension of Subspace (a Notion of ldquoSizerdquo)

                bull Number of Elements in a Basis (Unique)

                Review of Linear Algebra (Cont)

                Dimension of Subspace (a Notion of ldquoSizerdquo)

                bull Number of Elements in a Basis (Unique)

                bull (Use Basis Above)

                bull eg dim of a line is 1

                bull eg dim of a plane is 2

                dd dim

                Review of Linear Algebra (Cont)

                Dimension of Subspace (a Notion of ldquoSizerdquo)

                bull Number of Elements in a Basis (Unique)

                bull (Use Basis Above)

                bull eg dim of a line is 1

                bull eg dim of a plane is 2

                bull Dimension is ldquoDegrees of Freedomrdquo

                (in Statistical Uses eg ANOVA)

                dd dim

                Review of Linear Algebra (Cont)

                Norm of a Vector

                bull in d 21

                21

                1

                2 xxxx td

                jj

                Review of Linear Algebra (Cont)

                Norm of a Vector

                bull in

                bull Idea length of the vector

                d 2121

                1

                2 xxxx td

                jj

                Review of Linear Algebra (Cont)

                Norm of a Vector

                bull in

                bull Idea length of the vector

                bull Note strange properties for high

                eg ldquolength of diagonal of unit cuberdquo =

                d 2121

                1

                2 xxxx td

                jj

                d

                d

                Review of Linear Algebra (Cont)

                Norm of a Vector (cont)

                bull Length Normalized Vector

                (has Length 1 thus on Surf of Unit Sphere

                amp is a Direction Vector)

                x

                x

                Review of Linear Algebra (Cont)

                Norm of a Vector (cont)

                bull Length Normalized Vector

                (has Length 1 thus on Surf of Unit Sphere

                amp is a Direction Vector)

                bull Define Distance as

                x

                x

                yxyxyxyxd t

                Review of Linear Algebra (Cont)

                Inner (Dot Scalar) Product

                bull for Vectors and

                yxyxyx td

                jjj

                1

                x y

                Review of Linear Algebra (Cont)

                Inner (Dot Scalar) Product

                bull for Vectors and

                bull Related to Norm via

                yxyxyx td

                jjj

                1

                x y

                21 xxx

                Review of Linear Algebra (Cont)

                Inner (Dot Scalar) Product (cont)

                bull measures ldquoangle between and rdquo as

                yyxx

                yx

                yx

                yxyxangle

                tt

                t

                11 cos

                cos

                x y

                Review of Linear Algebra (Cont)

                Inner (Dot Scalar) Product (cont)

                bull measures ldquoangle between and rdquo as

                bull key to Orthogonality ie Perpendiculrsquoty

                if and only if

                yyxx

                yx

                yx

                yxyxangle

                tt

                t

                11 cos

                cos

                x y

                yx 0 yx

                Review of Linear Algebra (Cont)

                Orthonormal Basis

                bull All Orthogonal to each other

                ie for

                bull All have Length 1

                ie for

                nvv 1

                1 ii vv

                0 ii vv ii

                ni 1

                Review of Linear Algebra (Cont)

                Orthonormal Basis (cont)

                bull Spectral Representation

                where

                nvv 1

                n

                iii vax

                1

                ii vxa

                Review of Linear Algebra (Cont)

                Orthonormal Basis (cont)

                bull Spectral Representation

                where

                (Coefficient is Inner Product Cool Notation)

                nvv 1

                n

                iii vax

                1

                ii vxa

                Review of Linear Algebra (Cont)

                Orthonormal Basis (cont)

                bull Spectral Representation

                where

                Check

                nvv 1

                n

                iii vax

                1

                ii vxa

                iii

                n

                iii

                n

                iiii avvavvavx

                1

                1

                Review of Linear Algebra (Cont)

                Orthonormal Basis (cont)

                bull Spectral Representation

                where

                Check

                bull Matrix Notation where ie

                For the Basis Matrix

                nvv 1

                n

                iii vax

                1

                ii vxa

                iii

                n

                iii

                n

                iiii avvavvavx

                1

                1

                aBx Bxa tt xBa t

                nvvB 1

                Review of Linear Algebra (Cont)

                Orthonormal Basis (cont)

                bull Spectral Representation

                where

                Check

                bull Matrix Notation where ie

                is called transform of

                (eg Fourier or Wavelet)

                nvv 1

                n

                iii vax

                1

                ii vxa

                iii

                n

                iii

                n

                iiii avvavvavx

                1

                1

                aBx Bxa tt xBa t

                xa

                Review of Linear Algebra (Cont)

                Parseval identity for

                in subsp genrsquod by o n basis

                x

                nvv 1

                2

                1

                22

                1

                2 aavxx

                n

                ii

                n

                ii

                Review of Linear Algebra (Cont)

                Parseval identity for

                in subsp genrsquod by o n basis

                bull Pythagorean theorem

                bull ldquoDecomposition of Energyrdquo

                bull ANOVA - sums of squares

                x

                nvv 1

                2

                1

                22

                1

                2 aavxx

                n

                ii

                n

                ii

                Review of Linear Algebra (Cont)

                Parseval identity for

                in subsp genrsquod by o n basis

                bull Pythagorean theorem

                bull ldquoDecomposition of Energyrdquo

                bull ANOVA - sums of squares

                bull Transform has same length as

                ie ldquorotation in rdquo

                x

                nvv 1

                2

                1

                22

                1

                2 aavxx

                n

                ii

                n

                ii

                a xd

                Projection of a Vector onto a Subspace

                bull Idea Member of that is Closest to

                (ie ldquoBest Approxrsquonrdquo)

                Review of Linear Algebra (Cont)x

                xV

                V

                Projection of a Vector onto a Subspace

                bull Idea Member of that is Closest to

                (ie ldquoBest Approxrsquonrdquo)

                bull Find that Solves

                (ldquoLeast Squaresrdquo)

                Review of Linear Algebra (Cont)x

                xV

                V

                VxPV vxVv

                min

                Projection of a Vector onto a Subspace

                bull Idea Member of that is Closest to

                (ie ldquoBest Approxrsquonrdquo)

                bull Find that Solves

                (ldquoLeast Squaresrdquo)

                bull For Inner Product (Hilbert) Space

                Exists and is Unique

                Review of Linear Algebra (Cont)x

                xV

                V

                VxPV vxVv

                min

                xPV

                Projection of a Vector onto a Subspace (cont)

                bull General Solution in for Basis Matrix

                Review of Linear Algebra (Cont)

                d VB

                xBBBBxP tVV

                tVVV

                1

                Projection of a Vector onto a Subspace (cont)

                bull General Solution in for Basis Matrix

                bull So Projrsquon Operator is Matrix Multrsquon

                (thus projection is another linear operation)

                Review of Linear Algebra (Cont)

                d VB

                xBBBBxP tVV

                tVVV

                1

                tVV

                tVVV BBBBP

                1

                Projection of a Vector onto a Subspace (cont)

                bull General Solution in for Basis Matrix

                bull So Projrsquon Operator is Matrix Multrsquon

                (thus projection is another linear operation)

                (note same operation underlies least squares)

                Review of Linear Algebra (Cont)

                d VB

                xBBBBxP tVV

                tVVV

                1

                tVV

                tVVV BBBBP

                1

                Review of Linear Algebra (Cont)

                Projection using Orthonormal Basis

                bull Basis Matrix is Orthonormal

                nnVtV IBB

                10

                01

                1

                111

                1

                1

                nnn

                n

                ntn

                t

                vvvv

                vvvv

                vv

                v

                v

                nvv 1

                Review of Linear Algebra (Cont)

                Projection using Orthonormal Basis

                bull Basis Matrix is Orthonormal

                bull So =

                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                (Recall Right Multrsquon)

                nnVtV IBB

                10

                01

                1

                111

                1

                1

                nnn

                n

                ntn

                t

                vvvv

                vvvv

                vv

                v

                v

                xBBxP tVVV

                x V

                nvv 1

                Review of Linear Algebra (Cont)

                Projection using Orthonormal Basis (cont)

                bull For Orthogonal Complement

                and

                V

                xPxPx VV 222xPxPx VV

                Review of Linear Algebra (Cont)

                Projection using Orthonormal Basis (cont)

                bull For Orthogonal Complement

                and

                bull Parseval Inequality

                V

                xPxPx VV 222xPxPx VV

                2

                1

                22

                1

                22 aavxxxP

                n

                ii

                n

                iiV

                Review of Linear Algebra (Cont)

                (Real) Unitary Matrices with

                bull Orthonormal Basis Matrix

                (So All of Above Applies)

                ddU IUU t

                Review of Linear Algebra (Cont)

                (Real) Unitary Matrices with

                bull Orthonormal Basis Matrix

                (So All of Above Applies)

                bull Note Transformrsquon is Distance Preserving

                ddU IUU t

                yxdyxyxyxUyUxUdn

                i ii 2

                1

                Review of Linear Algebra (Cont)

                (Real) Unitary Matrices with

                bull Orthonormal Basis Matrix

                (So All of Above Applies)

                bull Note Transformrsquon is Distance Preserving

                bull Lin Trans (Mult by ) is ~ Rotation

                bull But also Includes ldquoMirror Imagesrdquo

                ddU IUU t

                yxdyxyxyxUyUxUdn

                i ii 2

                1

                U

                Review of Linear Algebra (Cont)

                Singular Value Decomposition (SVD)

                For a Matrix

                Find

                ndX

                Review of Linear Algebra (Cont)

                Singular Value Decomposition (SVD)

                For a Matrix

                Find a Diagonal Matrix

                with Entries

                called Singular Values

                ndX

                ndS

                )min(1 ndss

                Review of Linear Algebra (Cont)

                Singular Value Decomposition (SVD)

                For a Matrix

                Find a Diagonal Matrix

                with Entries

                called Singular Values

                And Unitary (Rotation) Matrices

                (recall )

                ndX

                ndS

                )min(1 ndss

                ddU nnV

                IVVUU tt

                Review of Linear Algebra (Cont)

                Singular Value Decomposition (SVD)

                For a Matrix

                Find a Diagonal Matrix

                with Entries

                called Singular Values

                And Unitary (Rotation) Matrices

                (recall )

                So That

                ndX

                ndS

                )min(1 ndss

                ddU nnV

                IVVUU tt tUSVX

                Review of Linear Algebra (Cont)

                Intuition behind Singular Value Decomposition

                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                vVSUvVSUvX tt

                Review of Linear Algebra (Cont)

                Intuition behind Singular Value Decomposition

                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                bull First rotate

                X

                vVSUvVSUvX tt

                Review of Linear Algebra (Cont)

                Intuition behind Singular Value Decomposition

                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                bull First rotate

                bull Second rescale coordinate axes (by )

                X

                vVSUvVSUvX tt

                is

                Review of Linear Algebra (Cont)

                Intuition behind Singular Value Decomposition

                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                bull First rotate

                bull Second rescale coordinate axes (by )

                bull Third rotate again

                X

                vVSUvVSUvX tt

                is

                Review of Linear Algebra (Cont)

                Intuition behind Singular Value Decomposition

                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                bull First rotate

                bull Second rescale coordinate axes (by )

                bull Third rotate again

                bull ie have diagonalized the transformation

                X

                vVSUvVSUvX tt

                is

                SVD Compact Representation

                Useful Labeling

                Singular Values in Increasing Order

                Review of Linear Algebra (Cont)

                )min(1 dnss

                SVD Compact Representation

                Useful Labeling

                Singular Values in Increasing Order

                Note singular values = 0 can be omitted

                (Since do ldquo0-Stretchingrdquo)

                Review of Linear Algebra (Cont)

                )min(1 dnss

                Review of Linear Algebra (Cont)

                r

                SVD Compact Representation

                Useful Labeling

                Singular Values in Increasing Order

                Note singular values = 0 can be omitted

                Let = of positive singular values

                )min(1 dnss

                Review of Linear Algebra (Cont)

                r

                SVD Compact Representation

                Useful Labeling

                Singular Values in Increasing Order

                Note singular values = 0 can be omitted

                Let = of positive singular values

                Then

                Where are truncations of

                trnrrrd VSUX

                VSU

                )min(1 dnss

                Review of Linear Algebra (Cont)

                SVD Full Representation

                =

                Graphics Display Assumes

                ndX ddU ndS nn

                tV

                nd

                Review of Linear Algebra (Cont)

                SVD Full Representation

                =

                Full Rank Basis Matrix

                ndX ddU ndS nn

                tV

                Review of Linear Algebra (Cont)

                SVD Full Representation

                =

                Full Rank Basis Matrix

                All 0s in Bottom

                ndX ddU ndS nn

                tV

                Review of Linear Algebra (Cont)

                SVD Reduced Representation

                =

                These Columns Get 0ed Out

                ndX ddU nnS nn

                tV

                nnd 0

                Review of Linear Algebra (Cont)

                SVD Reduced Representation

                =ndX ndU nnS nn

                tV

                Review of Linear Algebra (Cont)

                SVD Reduced Representation

                =

                Also Some of These May be 0

                ndX ndU nnS nn

                tV

                Review of Linear Algebra (Cont)

                SVD Compact Representation

                = ndX rdU

                rrS nrtV

                0

                Review of Linear Algebra (Cont)

                SVD Compact Representation

                =

                These Get 0ed Out

                ndX rdU

                rrS nrtV

                0

                Review of Linear Algebra (Cont)

                SVD Compact Representation

                = ndX rdU

                rrS nrtV

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition

                For a (Symmetric) Square Matrix

                Find

                ddX

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition

                For a (Symmetric) Square Matrix

                Find a Diagonal Matrix

                Called Eigenvalues

                Convenient Ordering

                ddX

                d

                D

                0

                01

                n 1

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition

                For a (Symmetric) Square Matrix

                Find a Diagonal Matrix

                And an Orthonormal Matrix

                (ie )

                ddX

                d

                D

                0

                01

                ddB

                ddtt IBBBB

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition

                For a (Symmetric) Square Matrix

                Find a Diagonal Matrix

                And an Orthonormal Matrix

                (ie )

                So that ie

                ddX

                d

                D

                0

                01

                ddB

                ddtt IBBBB

                DBBX tBDBX

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                (looks similar)

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                Complex (uses )

                VU

                1i

                Review of Linear Algebra (Cont)

                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                VU

                X

                BVU

                1i

                • Participant Presentations
                • PCA to find clusters
                • Statistical Smoothing
                • Kernel Density Estimation
                • Scatterplot Smoothing
                • Statistical Smoothing (2)
                • SiZer Background
                • SiZer Background (2)
                • SiZer Background (3)
                • SiZer Background (4)
                • SiZer Background (5)
                • SiZer Overview
                • PCA to find clusters (2)
                • PCA to find clusters (3)
                • PCA to find clusters (4)
                • PCA to find clusters (5)
                • PCA to find clusters (6)
                • PCA to find clusters (7)
                • Recall Yeast Cell Cycle Data
                • Yeast Cell Cycle Data FDA View
                • Yeast Cell Cycle Data FDA View (2)
                • Yeast Cell Cycle Data FDA View (3)
                • Yeast Cell Cycles Freq 2 Proj
                • Frequency 2 Analysis
                • Frequency 2 Analysis (2)
                • Frequency 2 Analysis (3)
                • Yeast Cell Cycle
                • SiZer Study of Distrsquon of Angles
                • Reclassification of Major Genes
                • Compare to Previous Classifrsquon
                • New Subpopulation View
                • New Subpopulation View (2)
                • Detailed Look at PCA
                • PCA Rediscovery ndash Renaming
                • PCA Rediscovery ndash Renaming (2)
                • PCA Rediscovery ndash Renaming (3)
                • PCA Rediscovery ndash Renaming (4)
                • PCA Rediscovery ndash Renaming (5)
                • An Interesting Historical Note
                • An Interesting Historical Note (2)
                • Detailed Look at PCA (2)
                • Detailed Look at PCA (3)
                • Review of Linear Algebra
                • Review of Linear Algebra (2)
                • Review of Linear Algebra (3)
                • Review of Linear Algebra (Cont)
                • Review of Linear Algebra (Cont) (2)
                • Review of Linear Algebra (Cont) (3)
                • Review of Linear Algebra (Cont) (4)
                • Review of Linear Algebra (Cont) (5)
                • Review of Linear Algebra (Cont) (6)
                • Review of Linear Algebra (Cont) (7)
                • Review of Linear Algebra (Cont) (8)
                • Review of Linear Algebra (Cont) (9)
                • Review of Linear Algebra (Cont) (10)
                • Review of Linear Algebra (Cont) (11)
                • Review of Linear Algebra (Cont) (12)
                • Review of Linear Algebra (Cont) (13)
                • Review of Linear Algebra (Cont) (14)
                • Review of Linear Algebra (Cont) (15)
                • Review of Linear Algebra (Cont) (16)
                • Review of Linear Algebra (Cont) (17)
                • Review of Linear Algebra (Cont) (18)
                • Review of Linear Algebra (Cont) (19)
                • Review of Linear Algebra (Cont) (20)
                • Review of Linear Algebra (Cont) (21)
                • Review of Linear Algebra (Cont) (22)
                • Review of Linear Algebra (Cont) (23)
                • Review of Linear Algebra (Cont) (24)
                • Review of Linear Algebra (Cont) (25)
                • Review of Linear Algebra (Cont) (26)
                • Review of Linear Algebra (Cont) (27)
                • Review of Linear Algebra (Cont) (28)
                • Review of Linear Algebra (Cont) (29)
                • Review of Linear Algebra (Cont) (30)
                • Review of Linear Algebra (Cont) (31)
                • Review of Linear Algebra (Cont) (32)
                • Review of Linear Algebra (Cont) (33)
                • Review of Linear Algebra (Cont) (34)
                • Review of Linear Algebra (Cont) (35)
                • Review of Linear Algebra (Cont) (36)
                • Review of Linear Algebra (Cont) (37)
                • Review of Linear Algebra (Cont) (38)
                • Review of Linear Algebra (Cont) (39)
                • Review of Linear Algebra (Cont) (40)
                • Review of Linear Algebra (Cont) (41)
                • Review of Linear Algebra (Cont) (42)
                • Review of Linear Algebra (Cont) (43)
                • Review of Linear Algebra (Cont) (44)
                • Review of Linear Algebra (Cont) (45)
                • Review of Linear Algebra (Cont) (46)
                • Review of Linear Algebra (Cont) (47)
                • Review of Linear Algebra (Cont) (48)
                • Review of Linear Algebra (Cont) (49)
                • Review of Linear Algebra (Cont) (50)
                • Review of Linear Algebra (Cont) (51)
                • Review of Linear Algebra (Cont) (52)
                • Review of Linear Algebra (Cont) (53)
                • Review of Linear Algebra (Cont) (54)
                • Review of Linear Algebra (Cont) (55)
                • Review of Linear Algebra (Cont) (56)
                • Review of Linear Algebra (Cont) (57)
                • Review of Linear Algebra (Cont) (58)
                • Review of Linear Algebra (Cont) (59)
                • Review of Linear Algebra (Cont) (60)
                • Review of Linear Algebra (Cont) (61)
                • Review of Linear Algebra (Cont) (62)
                • Review of Linear Algebra (Cont) (63)
                • Review of Linear Algebra (Cont) (64)
                • Review of Linear Algebra (Cont) (65)
                • Review of Linear Algebra (Cont) (66)
                • Review of Linear Algebra (Cont) (67)
                • Review of Linear Algebra (Cont) (68)
                • Review of Linear Algebra (Cont) (69)
                • Review of Linear Algebra (Cont) (70)
                • Review of Linear Algebra (Cont) (71)
                • Review of Linear Algebra (Cont) (72)
                • Review of Linear Algebra (Cont) (73)
                • Review of Linear Algebra (Cont) (74)
                • Review of Linear Algebra (Cont) (75)
                • Review of Linear Algebra (Cont) (76)
                • Review of Linear Algebra (Cont) (77)
                • Review of Linear Algebra (Cont) (78)
                • Review of Linear Algebra (Cont) (79)
                • Review of Linear Algebra (Cont) (80)
                • Review of Linear Algebra (Cont) (81)

                  SiZer BackgroundSiZer analysis of British Incomes data bull Oversmoothed Only one mode bull Medium smoothed Two modes

                  statistically significant

                  Confirmed by Schmitz amp Marron (1992)bull Undersmoothed many noise

                  wiggles not significant

                  Again all are correct

                  just different scales

                  SiZer BackgroundScale Space and Kernel Choice

                  ie Shape of Window

                  Compelling Answer Gaussian

                  Only ldquoVariation Diminishingrdquo Kernel Shape

                  I e Modes decreases with bandwidth h

                  Lindebergh (1994)

                  Chaudhuri amp Marron (2000)

                  SiZer BackgroundRecall

                  Hidalgo

                  Stamps

                  Data

                  >

                  SiZer Overview

                  Would you like to try smoothing amp

                  SiZer

                  bull Marron Software Website as Before

                  bull In ldquoSmoothingrdquo Directory

                  ndash kdeSMm

                  ndash nprSMm

                  ndash sizerSMm

                  bull Recall ldquogtgt help sizerSMrdquo for

                  usage

                  PCA to find clustersReturn to PCA of Mass Flux Data

                  PCA to find clustersSiZer analysis of Mass Flux PC1

                  PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                  PCA to find clustersSiZer analysis of Mass Flux PC1

                  Also in Curvature

                  PCA to find clustersSiZer analysis of Mass Flux PC1

                  And in Other Comprsquos

                  PCA to find clustersSiZer analysis of Mass Flux PC1

                  Conclusion

                  bull Found 3 significant clusters

                  bull Correspond to 3 known ldquocloud typesrdquo

                  bull Worth deeper investigation

                  Recall Yeast Cell Cycle Data

                  bull ldquoGene Expressionrdquo ndash Micro-array data

                  bull Data (after major preprocessing) Expression ldquolevelrdquo of

                  bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                  10s)bull Interesting statistical issue

                  High Dimension Low Sample Size data

                  (HDLSS)

                  Yeast Cell Cycle Data FDA View

                  Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                  Yeast Cell Cycle Data FDA View

                  Periodic genes

                  Naiumlve

                  approach

                  Simple PCA

                  Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                  ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                  ldquovariationrdquobull PCA finds ldquodirections of maximal

                  variationrdquobull Often but not always same as

                  ldquointeresting directionsrdquobull Here need better approach to study

                  periodicities

                  Yeast Cell Cycles Freq 2 Proj

                  PCA on

                  Freq 2

                  Periodic

                  Component

                  Of Data

                  Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                  cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                  Approach from Zhao Marron amp Wells (2004)

                  Frequency 2 Analysis

                  Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                  cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                  classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                  notable differencesbull Now try to improve ldquophase classificationrdquo

                  Yeast Cell CycleRevisit ldquophase classificationrdquo

                  approachbull Use outer 200 genes

                  (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                  (finds significant bumps etc in histogram)

                  bull Carefully redrew boundariesbull Check by studying kde angles

                  SiZer Study of Distrsquon of Angles

                  Reclassification of Major Genes

                  Compare to Previous Classifrsquon

                  New Subpopulation View

                  New Subpopulation View

                  NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                  Detailed Look at PCA

                  Now Study ldquoFolklorerdquo More Carefully

                  bull BackGround

                  bull History

                  bull Underpinnings

                  (Mathematical amp Computational)

                  Good Overall Reference Jolliffe (2002)

                  PCA Rediscovery ndash Renaming

                  Statistics Principal Component Analysis (PCA)

                  PCA Rediscovery ndash Renaming

                  Statistics Principal Component Analysis (PCA)

                  Social Sciences Factor Analysis (PCA is a subset)

                  PCA Rediscovery ndash Renaming

                  Statistics Principal Component Analysis (PCA)

                  Social Sciences Factor Analysis (PCA is a subset)

                  Probability Electrical EngKarhunen ndash Loeve expansion

                  PCA Rediscovery ndash Renaming

                  Statistics Principal Component Analysis (PCA)

                  Social Sciences Factor Analysis (PCA is a subset)

                  Probability Electrical EngKarhunen ndash Loeve expansion

                  Applied MathematicsProper Orthogonal Decomposition (POD)

                  PCA Rediscovery ndash Renaming

                  Statistics Principal Component Analysis (PCA)

                  Social Sciences Factor Analysis (PCA is a subset)

                  Probability Electrical EngKarhunen ndash Loeve expansion

                  Applied MathematicsProper Orthogonal Decomposition (POD)

                  Geo-Sciences Empirical Orthogonal Functions (EOF)

                  An Interesting Historical Note

                  The 1st () application of PCA to Functional

                  Data Analysis

                  An Interesting Historical Note

                  The 1st () application of PCA to Functional

                  Data Analysis

                  Rao (1958)

                  1st Paper with ldquoCurves as Data Objectsrdquo

                  viewpoint

                  Detailed Look at PCA

                  Three Important (amp Interesting) Viewpoints

                  1 Mathematics

                  2 Numerics

                  3 Statistics

                  Goal Study Interrelationships

                  Detailed Look at PCA

                  Three Important (amp Interesting) Viewpoints

                  1 Mathematics

                  2 Numerics

                  3 Statistics

                  1st Review Linear Alg and Multivar Prob

                  Review of Linear Algebra

                  Vector Space

                  bull set of ldquovectorsrdquo

                  bull and ldquoscalarsrdquo (coefficients)

                  xa

                  Review of Linear Algebra

                  Vector Space

                  bull set of ldquovectorsrdquo

                  bull and ldquoscalarsrdquo (coefficients)

                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                  ( in space)

                  xa

                  i

                  ii xa

                  Review of Linear Algebra

                  Vector Space

                  bull set of ldquovectorsrdquo

                  bull and ldquoscalarsrdquo (coefficients)

                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                  ( in space)

                  eg

                  ldquo dim Euclidrsquon spacerdquo

                  xa

                  i

                  ii xa

                  d

                  d

                  d xx

                  x

                  x

                  x 1

                  1

                  d

                  Review of Linear Algebra (Cont)

                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                  Review of Linear Algebra (Cont)

                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                  Review of Linear Algebra (Cont)

                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                  Note Planes not Through the Origin

                  are not Subspaces

                  (Do not Contain )00 x

                  Review of Linear Algebra (Cont)

                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                  (all Linear Combos of them =

                  = Containing Hyperplane

                  through Origin)

                  Review of Linear Algebra (Cont)

                  Basis of Subspace Set of Vectors that

                  bull Span ie Everything is a Lin Com of them

                  bull are Linearly Indeprsquot ie Lin Com is Unique

                  Review of Linear Algebra (Cont)

                  Basis of Subspace Set of Vectors that

                  bull Span ie Everything is a Lin Com of them

                  bull are Linearly Indeprsquot ie Lin Com is Unique

                  bull eg ldquoUnit Vector Basisrdquo

                  bull Since

                  d

                  1

                  0

                  0

                  0

                  1

                  0

                  0

                  0

                  1

                  1

                  0

                  0

                  0

                  1

                  0

                  0

                  0

                  1

                  212

                  1

                  d

                  d

                  xxx

                  x

                  x

                  x

                  Review of Linear Algebra (Cont)

                  Basis Matrix of subspace of

                  Given a basis

                  create matrix of columns

                  dnvv 1

                  nddnd

                  n

                  n

                  vv

                  vv

                  vvB

                  1

                  111

                  1

                  Review of Linear Algebra (Cont)

                  Then linear combo is a matrix multiplicatrsquon

                  where

                  n

                  iii aBva

                  1

                  na

                  a

                  a 1

                  Review of Linear Algebra (Cont)

                  Then linear combo is a matrix multiplicatrsquon

                  where

                  Note Right Multiplication Gives

                  Linear Combination of Column Vectors

                  n

                  iii aBva

                  1

                  na

                  a

                  a 1

                  Review of Linear Algebra (Cont)

                  Then linear combo is a matrix multiplicatrsquon

                  where

                  Check sizes

                  n

                  iii aBva

                  1

                  na

                  a

                  a 1

                  )1()(1 nndd

                  Review of Linear Algebra (Cont)

                  Aside on Matrix Multiplication (linear transformatrsquon)

                  For matrices

                  Define the Matrix Product

                  mkk

                  m

                  aa

                  aa

                  A

                  1

                  111

                  nmm

                  n

                  bb

                  bb

                  B

                  1

                  111

                  m

                  iniik

                  m

                  iiik

                  m

                  inii

                  m

                  iii

                  baba

                  baba

                  AB

                  1

                  11

                  11

                  111

                  Review of Linear Algebra (Cont)

                  Aside on Matrix Multiplication (linear transformatrsquon)

                  For matrices

                  Define the Matrix Product

                  (Inner Products of Rows With Columns )

                  mkk

                  m

                  aa

                  aa

                  A

                  1

                  111

                  nmm

                  n

                  bb

                  bb

                  B

                  1

                  111

                  m

                  iniik

                  m

                  iiik

                  m

                  inii

                  m

                  iii

                  baba

                  baba

                  AB

                  1

                  11

                  11

                  111

                  A B

                  Review of Linear Algebra (Cont)

                  Aside on Matrix Multiplication (linear transformatrsquon)

                  For matrices

                  Define the Matrix Product

                  (Inner Products of Rows With Columns )

                  (Composition of Linear Transformations)

                  mkk

                  m

                  aa

                  aa

                  A

                  1

                  111

                  nmm

                  n

                  bb

                  bb

                  B

                  1

                  111

                  m

                  iniik

                  m

                  iiik

                  m

                  inii

                  m

                  iii

                  baba

                  baba

                  AB

                  1

                  11

                  11

                  111

                  A B

                  Review of Linear Algebra (Cont)

                  Aside on Matrix Multiplication (linear transformatrsquon)

                  For matrices

                  Define the Matrix Product

                  (Inner Products of Rows With Columns )

                  (Composition of Linear Transformations)

                  Often Useful to Check Sizes

                  mkk

                  m

                  aa

                  aa

                  A

                  1

                  111

                  nmm

                  n

                  bb

                  bb

                  B

                  1

                  111

                  m

                  iniik

                  m

                  iiik

                  m

                  inii

                  m

                  iii

                  baba

                  baba

                  AB

                  1

                  11

                  11

                  111

                  nmmknk

                  A B

                  Review of Linear Algebra (Cont)

                  Aside on Matrix Multiplication (linear transformatrsquon)

                  For matrices

                  Define the Matrix Product

                  (Inner Products of Rows With Columns )

                  (Composition of Linear Transformations)

                  Often Useful to Check Sizes

                  mkk

                  m

                  aa

                  aa

                  A

                  1

                  111

                  nmm

                  n

                  bb

                  bb

                  B

                  1

                  111

                  m

                  iniik

                  m

                  iiik

                  m

                  inii

                  m

                  iii

                  baba

                  baba

                  AB

                  1

                  11

                  11

                  111

                  nmmknk

                  A B

                  Review of Linear Algebra (Cont)

                  Matrix Trace

                  bull For a Square Matrix

                  bull Define

                  mmm

                  m

                  aa

                  aa

                  A

                  1

                  111

                  m

                  iiiaAtr

                  1)(

                  Review of Linear Algebra (Cont)

                  Matrix Trace

                  bull For a Square Matrix

                  bull Define

                  bull Trace Commutes with Matrix Multiplication

                  mmm

                  m

                  aa

                  aa

                  A

                  1

                  111

                  m

                  iiiaAtr

                  1)(

                  BAtrABtr

                  Review of Linear Algebra (Cont)

                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                  bull Number of Elements in a Basis (Unique)

                  Review of Linear Algebra (Cont)

                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                  bull Number of Elements in a Basis (Unique)

                  bull (Use Basis Above)

                  bull eg dim of a line is 1

                  bull eg dim of a plane is 2

                  dd dim

                  Review of Linear Algebra (Cont)

                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                  bull Number of Elements in a Basis (Unique)

                  bull (Use Basis Above)

                  bull eg dim of a line is 1

                  bull eg dim of a plane is 2

                  bull Dimension is ldquoDegrees of Freedomrdquo

                  (in Statistical Uses eg ANOVA)

                  dd dim

                  Review of Linear Algebra (Cont)

                  Norm of a Vector

                  bull in d 21

                  21

                  1

                  2 xxxx td

                  jj

                  Review of Linear Algebra (Cont)

                  Norm of a Vector

                  bull in

                  bull Idea length of the vector

                  d 2121

                  1

                  2 xxxx td

                  jj

                  Review of Linear Algebra (Cont)

                  Norm of a Vector

                  bull in

                  bull Idea length of the vector

                  bull Note strange properties for high

                  eg ldquolength of diagonal of unit cuberdquo =

                  d 2121

                  1

                  2 xxxx td

                  jj

                  d

                  d

                  Review of Linear Algebra (Cont)

                  Norm of a Vector (cont)

                  bull Length Normalized Vector

                  (has Length 1 thus on Surf of Unit Sphere

                  amp is a Direction Vector)

                  x

                  x

                  Review of Linear Algebra (Cont)

                  Norm of a Vector (cont)

                  bull Length Normalized Vector

                  (has Length 1 thus on Surf of Unit Sphere

                  amp is a Direction Vector)

                  bull Define Distance as

                  x

                  x

                  yxyxyxyxd t

                  Review of Linear Algebra (Cont)

                  Inner (Dot Scalar) Product

                  bull for Vectors and

                  yxyxyx td

                  jjj

                  1

                  x y

                  Review of Linear Algebra (Cont)

                  Inner (Dot Scalar) Product

                  bull for Vectors and

                  bull Related to Norm via

                  yxyxyx td

                  jjj

                  1

                  x y

                  21 xxx

                  Review of Linear Algebra (Cont)

                  Inner (Dot Scalar) Product (cont)

                  bull measures ldquoangle between and rdquo as

                  yyxx

                  yx

                  yx

                  yxyxangle

                  tt

                  t

                  11 cos

                  cos

                  x y

                  Review of Linear Algebra (Cont)

                  Inner (Dot Scalar) Product (cont)

                  bull measures ldquoangle between and rdquo as

                  bull key to Orthogonality ie Perpendiculrsquoty

                  if and only if

                  yyxx

                  yx

                  yx

                  yxyxangle

                  tt

                  t

                  11 cos

                  cos

                  x y

                  yx 0 yx

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis

                  bull All Orthogonal to each other

                  ie for

                  bull All have Length 1

                  ie for

                  nvv 1

                  1 ii vv

                  0 ii vv ii

                  ni 1

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis (cont)

                  bull Spectral Representation

                  where

                  nvv 1

                  n

                  iii vax

                  1

                  ii vxa

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis (cont)

                  bull Spectral Representation

                  where

                  (Coefficient is Inner Product Cool Notation)

                  nvv 1

                  n

                  iii vax

                  1

                  ii vxa

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis (cont)

                  bull Spectral Representation

                  where

                  Check

                  nvv 1

                  n

                  iii vax

                  1

                  ii vxa

                  iii

                  n

                  iii

                  n

                  iiii avvavvavx

                  1

                  1

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis (cont)

                  bull Spectral Representation

                  where

                  Check

                  bull Matrix Notation where ie

                  For the Basis Matrix

                  nvv 1

                  n

                  iii vax

                  1

                  ii vxa

                  iii

                  n

                  iii

                  n

                  iiii avvavvavx

                  1

                  1

                  aBx Bxa tt xBa t

                  nvvB 1

                  Review of Linear Algebra (Cont)

                  Orthonormal Basis (cont)

                  bull Spectral Representation

                  where

                  Check

                  bull Matrix Notation where ie

                  is called transform of

                  (eg Fourier or Wavelet)

                  nvv 1

                  n

                  iii vax

                  1

                  ii vxa

                  iii

                  n

                  iii

                  n

                  iiii avvavvavx

                  1

                  1

                  aBx Bxa tt xBa t

                  xa

                  Review of Linear Algebra (Cont)

                  Parseval identity for

                  in subsp genrsquod by o n basis

                  x

                  nvv 1

                  2

                  1

                  22

                  1

                  2 aavxx

                  n

                  ii

                  n

                  ii

                  Review of Linear Algebra (Cont)

                  Parseval identity for

                  in subsp genrsquod by o n basis

                  bull Pythagorean theorem

                  bull ldquoDecomposition of Energyrdquo

                  bull ANOVA - sums of squares

                  x

                  nvv 1

                  2

                  1

                  22

                  1

                  2 aavxx

                  n

                  ii

                  n

                  ii

                  Review of Linear Algebra (Cont)

                  Parseval identity for

                  in subsp genrsquod by o n basis

                  bull Pythagorean theorem

                  bull ldquoDecomposition of Energyrdquo

                  bull ANOVA - sums of squares

                  bull Transform has same length as

                  ie ldquorotation in rdquo

                  x

                  nvv 1

                  2

                  1

                  22

                  1

                  2 aavxx

                  n

                  ii

                  n

                  ii

                  a xd

                  Projection of a Vector onto a Subspace

                  bull Idea Member of that is Closest to

                  (ie ldquoBest Approxrsquonrdquo)

                  Review of Linear Algebra (Cont)x

                  xV

                  V

                  Projection of a Vector onto a Subspace

                  bull Idea Member of that is Closest to

                  (ie ldquoBest Approxrsquonrdquo)

                  bull Find that Solves

                  (ldquoLeast Squaresrdquo)

                  Review of Linear Algebra (Cont)x

                  xV

                  V

                  VxPV vxVv

                  min

                  Projection of a Vector onto a Subspace

                  bull Idea Member of that is Closest to

                  (ie ldquoBest Approxrsquonrdquo)

                  bull Find that Solves

                  (ldquoLeast Squaresrdquo)

                  bull For Inner Product (Hilbert) Space

                  Exists and is Unique

                  Review of Linear Algebra (Cont)x

                  xV

                  V

                  VxPV vxVv

                  min

                  xPV

                  Projection of a Vector onto a Subspace (cont)

                  bull General Solution in for Basis Matrix

                  Review of Linear Algebra (Cont)

                  d VB

                  xBBBBxP tVV

                  tVVV

                  1

                  Projection of a Vector onto a Subspace (cont)

                  bull General Solution in for Basis Matrix

                  bull So Projrsquon Operator is Matrix Multrsquon

                  (thus projection is another linear operation)

                  Review of Linear Algebra (Cont)

                  d VB

                  xBBBBxP tVV

                  tVVV

                  1

                  tVV

                  tVVV BBBBP

                  1

                  Projection of a Vector onto a Subspace (cont)

                  bull General Solution in for Basis Matrix

                  bull So Projrsquon Operator is Matrix Multrsquon

                  (thus projection is another linear operation)

                  (note same operation underlies least squares)

                  Review of Linear Algebra (Cont)

                  d VB

                  xBBBBxP tVV

                  tVVV

                  1

                  tVV

                  tVVV BBBBP

                  1

                  Review of Linear Algebra (Cont)

                  Projection using Orthonormal Basis

                  bull Basis Matrix is Orthonormal

                  nnVtV IBB

                  10

                  01

                  1

                  111

                  1

                  1

                  nnn

                  n

                  ntn

                  t

                  vvvv

                  vvvv

                  vv

                  v

                  v

                  nvv 1

                  Review of Linear Algebra (Cont)

                  Projection using Orthonormal Basis

                  bull Basis Matrix is Orthonormal

                  bull So =

                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                  (Recall Right Multrsquon)

                  nnVtV IBB

                  10

                  01

                  1

                  111

                  1

                  1

                  nnn

                  n

                  ntn

                  t

                  vvvv

                  vvvv

                  vv

                  v

                  v

                  xBBxP tVVV

                  x V

                  nvv 1

                  Review of Linear Algebra (Cont)

                  Projection using Orthonormal Basis (cont)

                  bull For Orthogonal Complement

                  and

                  V

                  xPxPx VV 222xPxPx VV

                  Review of Linear Algebra (Cont)

                  Projection using Orthonormal Basis (cont)

                  bull For Orthogonal Complement

                  and

                  bull Parseval Inequality

                  V

                  xPxPx VV 222xPxPx VV

                  2

                  1

                  22

                  1

                  22 aavxxxP

                  n

                  ii

                  n

                  iiV

                  Review of Linear Algebra (Cont)

                  (Real) Unitary Matrices with

                  bull Orthonormal Basis Matrix

                  (So All of Above Applies)

                  ddU IUU t

                  Review of Linear Algebra (Cont)

                  (Real) Unitary Matrices with

                  bull Orthonormal Basis Matrix

                  (So All of Above Applies)

                  bull Note Transformrsquon is Distance Preserving

                  ddU IUU t

                  yxdyxyxyxUyUxUdn

                  i ii 2

                  1

                  Review of Linear Algebra (Cont)

                  (Real) Unitary Matrices with

                  bull Orthonormal Basis Matrix

                  (So All of Above Applies)

                  bull Note Transformrsquon is Distance Preserving

                  bull Lin Trans (Mult by ) is ~ Rotation

                  bull But also Includes ldquoMirror Imagesrdquo

                  ddU IUU t

                  yxdyxyxyxUyUxUdn

                  i ii 2

                  1

                  U

                  Review of Linear Algebra (Cont)

                  Singular Value Decomposition (SVD)

                  For a Matrix

                  Find

                  ndX

                  Review of Linear Algebra (Cont)

                  Singular Value Decomposition (SVD)

                  For a Matrix

                  Find a Diagonal Matrix

                  with Entries

                  called Singular Values

                  ndX

                  ndS

                  )min(1 ndss

                  Review of Linear Algebra (Cont)

                  Singular Value Decomposition (SVD)

                  For a Matrix

                  Find a Diagonal Matrix

                  with Entries

                  called Singular Values

                  And Unitary (Rotation) Matrices

                  (recall )

                  ndX

                  ndS

                  )min(1 ndss

                  ddU nnV

                  IVVUU tt

                  Review of Linear Algebra (Cont)

                  Singular Value Decomposition (SVD)

                  For a Matrix

                  Find a Diagonal Matrix

                  with Entries

                  called Singular Values

                  And Unitary (Rotation) Matrices

                  (recall )

                  So That

                  ndX

                  ndS

                  )min(1 ndss

                  ddU nnV

                  IVVUU tt tUSVX

                  Review of Linear Algebra (Cont)

                  Intuition behind Singular Value Decomposition

                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                  vVSUvVSUvX tt

                  Review of Linear Algebra (Cont)

                  Intuition behind Singular Value Decomposition

                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                  bull First rotate

                  X

                  vVSUvVSUvX tt

                  Review of Linear Algebra (Cont)

                  Intuition behind Singular Value Decomposition

                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                  bull First rotate

                  bull Second rescale coordinate axes (by )

                  X

                  vVSUvVSUvX tt

                  is

                  Review of Linear Algebra (Cont)

                  Intuition behind Singular Value Decomposition

                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                  bull First rotate

                  bull Second rescale coordinate axes (by )

                  bull Third rotate again

                  X

                  vVSUvVSUvX tt

                  is

                  Review of Linear Algebra (Cont)

                  Intuition behind Singular Value Decomposition

                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                  bull First rotate

                  bull Second rescale coordinate axes (by )

                  bull Third rotate again

                  bull ie have diagonalized the transformation

                  X

                  vVSUvVSUvX tt

                  is

                  SVD Compact Representation

                  Useful Labeling

                  Singular Values in Increasing Order

                  Review of Linear Algebra (Cont)

                  )min(1 dnss

                  SVD Compact Representation

                  Useful Labeling

                  Singular Values in Increasing Order

                  Note singular values = 0 can be omitted

                  (Since do ldquo0-Stretchingrdquo)

                  Review of Linear Algebra (Cont)

                  )min(1 dnss

                  Review of Linear Algebra (Cont)

                  r

                  SVD Compact Representation

                  Useful Labeling

                  Singular Values in Increasing Order

                  Note singular values = 0 can be omitted

                  Let = of positive singular values

                  )min(1 dnss

                  Review of Linear Algebra (Cont)

                  r

                  SVD Compact Representation

                  Useful Labeling

                  Singular Values in Increasing Order

                  Note singular values = 0 can be omitted

                  Let = of positive singular values

                  Then

                  Where are truncations of

                  trnrrrd VSUX

                  VSU

                  )min(1 dnss

                  Review of Linear Algebra (Cont)

                  SVD Full Representation

                  =

                  Graphics Display Assumes

                  ndX ddU ndS nn

                  tV

                  nd

                  Review of Linear Algebra (Cont)

                  SVD Full Representation

                  =

                  Full Rank Basis Matrix

                  ndX ddU ndS nn

                  tV

                  Review of Linear Algebra (Cont)

                  SVD Full Representation

                  =

                  Full Rank Basis Matrix

                  All 0s in Bottom

                  ndX ddU ndS nn

                  tV

                  Review of Linear Algebra (Cont)

                  SVD Reduced Representation

                  =

                  These Columns Get 0ed Out

                  ndX ddU nnS nn

                  tV

                  nnd 0

                  Review of Linear Algebra (Cont)

                  SVD Reduced Representation

                  =ndX ndU nnS nn

                  tV

                  Review of Linear Algebra (Cont)

                  SVD Reduced Representation

                  =

                  Also Some of These May be 0

                  ndX ndU nnS nn

                  tV

                  Review of Linear Algebra (Cont)

                  SVD Compact Representation

                  = ndX rdU

                  rrS nrtV

                  0

                  Review of Linear Algebra (Cont)

                  SVD Compact Representation

                  =

                  These Get 0ed Out

                  ndX rdU

                  rrS nrtV

                  0

                  Review of Linear Algebra (Cont)

                  SVD Compact Representation

                  = ndX rdU

                  rrS nrtV

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition

                  For a (Symmetric) Square Matrix

                  Find

                  ddX

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition

                  For a (Symmetric) Square Matrix

                  Find a Diagonal Matrix

                  Called Eigenvalues

                  Convenient Ordering

                  ddX

                  d

                  D

                  0

                  01

                  n 1

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition

                  For a (Symmetric) Square Matrix

                  Find a Diagonal Matrix

                  And an Orthonormal Matrix

                  (ie )

                  ddX

                  d

                  D

                  0

                  01

                  ddB

                  ddtt IBBBB

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition

                  For a (Symmetric) Square Matrix

                  Find a Diagonal Matrix

                  And an Orthonormal Matrix

                  (ie )

                  So that ie

                  ddX

                  d

                  D

                  0

                  01

                  ddB

                  ddtt IBBBB

                  DBBX tBDBX

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                  (looks similar)

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                  Complex (uses )

                  VU

                  1i

                  Review of Linear Algebra (Cont)

                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                  VU

                  X

                  BVU

                  1i

                  • Participant Presentations
                  • PCA to find clusters
                  • Statistical Smoothing
                  • Kernel Density Estimation
                  • Scatterplot Smoothing
                  • Statistical Smoothing (2)
                  • SiZer Background
                  • SiZer Background (2)
                  • SiZer Background (3)
                  • SiZer Background (4)
                  • SiZer Background (5)
                  • SiZer Overview
                  • PCA to find clusters (2)
                  • PCA to find clusters (3)
                  • PCA to find clusters (4)
                  • PCA to find clusters (5)
                  • PCA to find clusters (6)
                  • PCA to find clusters (7)
                  • Recall Yeast Cell Cycle Data
                  • Yeast Cell Cycle Data FDA View
                  • Yeast Cell Cycle Data FDA View (2)
                  • Yeast Cell Cycle Data FDA View (3)
                  • Yeast Cell Cycles Freq 2 Proj
                  • Frequency 2 Analysis
                  • Frequency 2 Analysis (2)
                  • Frequency 2 Analysis (3)
                  • Yeast Cell Cycle
                  • SiZer Study of Distrsquon of Angles
                  • Reclassification of Major Genes
                  • Compare to Previous Classifrsquon
                  • New Subpopulation View
                  • New Subpopulation View (2)
                  • Detailed Look at PCA
                  • PCA Rediscovery ndash Renaming
                  • PCA Rediscovery ndash Renaming (2)
                  • PCA Rediscovery ndash Renaming (3)
                  • PCA Rediscovery ndash Renaming (4)
                  • PCA Rediscovery ndash Renaming (5)
                  • An Interesting Historical Note
                  • An Interesting Historical Note (2)
                  • Detailed Look at PCA (2)
                  • Detailed Look at PCA (3)
                  • Review of Linear Algebra
                  • Review of Linear Algebra (2)
                  • Review of Linear Algebra (3)
                  • Review of Linear Algebra (Cont)
                  • Review of Linear Algebra (Cont) (2)
                  • Review of Linear Algebra (Cont) (3)
                  • Review of Linear Algebra (Cont) (4)
                  • Review of Linear Algebra (Cont) (5)
                  • Review of Linear Algebra (Cont) (6)
                  • Review of Linear Algebra (Cont) (7)
                  • Review of Linear Algebra (Cont) (8)
                  • Review of Linear Algebra (Cont) (9)
                  • Review of Linear Algebra (Cont) (10)
                  • Review of Linear Algebra (Cont) (11)
                  • Review of Linear Algebra (Cont) (12)
                  • Review of Linear Algebra (Cont) (13)
                  • Review of Linear Algebra (Cont) (14)
                  • Review of Linear Algebra (Cont) (15)
                  • Review of Linear Algebra (Cont) (16)
                  • Review of Linear Algebra (Cont) (17)
                  • Review of Linear Algebra (Cont) (18)
                  • Review of Linear Algebra (Cont) (19)
                  • Review of Linear Algebra (Cont) (20)
                  • Review of Linear Algebra (Cont) (21)
                  • Review of Linear Algebra (Cont) (22)
                  • Review of Linear Algebra (Cont) (23)
                  • Review of Linear Algebra (Cont) (24)
                  • Review of Linear Algebra (Cont) (25)
                  • Review of Linear Algebra (Cont) (26)
                  • Review of Linear Algebra (Cont) (27)
                  • Review of Linear Algebra (Cont) (28)
                  • Review of Linear Algebra (Cont) (29)
                  • Review of Linear Algebra (Cont) (30)
                  • Review of Linear Algebra (Cont) (31)
                  • Review of Linear Algebra (Cont) (32)
                  • Review of Linear Algebra (Cont) (33)
                  • Review of Linear Algebra (Cont) (34)
                  • Review of Linear Algebra (Cont) (35)
                  • Review of Linear Algebra (Cont) (36)
                  • Review of Linear Algebra (Cont) (37)
                  • Review of Linear Algebra (Cont) (38)
                  • Review of Linear Algebra (Cont) (39)
                  • Review of Linear Algebra (Cont) (40)
                  • Review of Linear Algebra (Cont) (41)
                  • Review of Linear Algebra (Cont) (42)
                  • Review of Linear Algebra (Cont) (43)
                  • Review of Linear Algebra (Cont) (44)
                  • Review of Linear Algebra (Cont) (45)
                  • Review of Linear Algebra (Cont) (46)
                  • Review of Linear Algebra (Cont) (47)
                  • Review of Linear Algebra (Cont) (48)
                  • Review of Linear Algebra (Cont) (49)
                  • Review of Linear Algebra (Cont) (50)
                  • Review of Linear Algebra (Cont) (51)
                  • Review of Linear Algebra (Cont) (52)
                  • Review of Linear Algebra (Cont) (53)
                  • Review of Linear Algebra (Cont) (54)
                  • Review of Linear Algebra (Cont) (55)
                  • Review of Linear Algebra (Cont) (56)
                  • Review of Linear Algebra (Cont) (57)
                  • Review of Linear Algebra (Cont) (58)
                  • Review of Linear Algebra (Cont) (59)
                  • Review of Linear Algebra (Cont) (60)
                  • Review of Linear Algebra (Cont) (61)
                  • Review of Linear Algebra (Cont) (62)
                  • Review of Linear Algebra (Cont) (63)
                  • Review of Linear Algebra (Cont) (64)
                  • Review of Linear Algebra (Cont) (65)
                  • Review of Linear Algebra (Cont) (66)
                  • Review of Linear Algebra (Cont) (67)
                  • Review of Linear Algebra (Cont) (68)
                  • Review of Linear Algebra (Cont) (69)
                  • Review of Linear Algebra (Cont) (70)
                  • Review of Linear Algebra (Cont) (71)
                  • Review of Linear Algebra (Cont) (72)
                  • Review of Linear Algebra (Cont) (73)
                  • Review of Linear Algebra (Cont) (74)
                  • Review of Linear Algebra (Cont) (75)
                  • Review of Linear Algebra (Cont) (76)
                  • Review of Linear Algebra (Cont) (77)
                  • Review of Linear Algebra (Cont) (78)
                  • Review of Linear Algebra (Cont) (79)
                  • Review of Linear Algebra (Cont) (80)
                  • Review of Linear Algebra (Cont) (81)

                    SiZer BackgroundScale Space and Kernel Choice

                    ie Shape of Window

                    Compelling Answer Gaussian

                    Only ldquoVariation Diminishingrdquo Kernel Shape

                    I e Modes decreases with bandwidth h

                    Lindebergh (1994)

                    Chaudhuri amp Marron (2000)

                    SiZer BackgroundRecall

                    Hidalgo

                    Stamps

                    Data

                    >

                    SiZer Overview

                    Would you like to try smoothing amp

                    SiZer

                    bull Marron Software Website as Before

                    bull In ldquoSmoothingrdquo Directory

                    ndash kdeSMm

                    ndash nprSMm

                    ndash sizerSMm

                    bull Recall ldquogtgt help sizerSMrdquo for

                    usage

                    PCA to find clustersReturn to PCA of Mass Flux Data

                    PCA to find clustersSiZer analysis of Mass Flux PC1

                    PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                    PCA to find clustersSiZer analysis of Mass Flux PC1

                    Also in Curvature

                    PCA to find clustersSiZer analysis of Mass Flux PC1

                    And in Other Comprsquos

                    PCA to find clustersSiZer analysis of Mass Flux PC1

                    Conclusion

                    bull Found 3 significant clusters

                    bull Correspond to 3 known ldquocloud typesrdquo

                    bull Worth deeper investigation

                    Recall Yeast Cell Cycle Data

                    bull ldquoGene Expressionrdquo ndash Micro-array data

                    bull Data (after major preprocessing) Expression ldquolevelrdquo of

                    bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                    10s)bull Interesting statistical issue

                    High Dimension Low Sample Size data

                    (HDLSS)

                    Yeast Cell Cycle Data FDA View

                    Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                    Yeast Cell Cycle Data FDA View

                    Periodic genes

                    Naiumlve

                    approach

                    Simple PCA

                    Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                    ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                    ldquovariationrdquobull PCA finds ldquodirections of maximal

                    variationrdquobull Often but not always same as

                    ldquointeresting directionsrdquobull Here need better approach to study

                    periodicities

                    Yeast Cell Cycles Freq 2 Proj

                    PCA on

                    Freq 2

                    Periodic

                    Component

                    Of Data

                    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                    Approach from Zhao Marron amp Wells (2004)

                    Frequency 2 Analysis

                    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                    classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                    notable differencesbull Now try to improve ldquophase classificationrdquo

                    Yeast Cell CycleRevisit ldquophase classificationrdquo

                    approachbull Use outer 200 genes

                    (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                    (finds significant bumps etc in histogram)

                    bull Carefully redrew boundariesbull Check by studying kde angles

                    SiZer Study of Distrsquon of Angles

                    Reclassification of Major Genes

                    Compare to Previous Classifrsquon

                    New Subpopulation View

                    New Subpopulation View

                    NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                    Detailed Look at PCA

                    Now Study ldquoFolklorerdquo More Carefully

                    bull BackGround

                    bull History

                    bull Underpinnings

                    (Mathematical amp Computational)

                    Good Overall Reference Jolliffe (2002)

                    PCA Rediscovery ndash Renaming

                    Statistics Principal Component Analysis (PCA)

                    PCA Rediscovery ndash Renaming

                    Statistics Principal Component Analysis (PCA)

                    Social Sciences Factor Analysis (PCA is a subset)

                    PCA Rediscovery ndash Renaming

                    Statistics Principal Component Analysis (PCA)

                    Social Sciences Factor Analysis (PCA is a subset)

                    Probability Electrical EngKarhunen ndash Loeve expansion

                    PCA Rediscovery ndash Renaming

                    Statistics Principal Component Analysis (PCA)

                    Social Sciences Factor Analysis (PCA is a subset)

                    Probability Electrical EngKarhunen ndash Loeve expansion

                    Applied MathematicsProper Orthogonal Decomposition (POD)

                    PCA Rediscovery ndash Renaming

                    Statistics Principal Component Analysis (PCA)

                    Social Sciences Factor Analysis (PCA is a subset)

                    Probability Electrical EngKarhunen ndash Loeve expansion

                    Applied MathematicsProper Orthogonal Decomposition (POD)

                    Geo-Sciences Empirical Orthogonal Functions (EOF)

                    An Interesting Historical Note

                    The 1st () application of PCA to Functional

                    Data Analysis

                    An Interesting Historical Note

                    The 1st () application of PCA to Functional

                    Data Analysis

                    Rao (1958)

                    1st Paper with ldquoCurves as Data Objectsrdquo

                    viewpoint

                    Detailed Look at PCA

                    Three Important (amp Interesting) Viewpoints

                    1 Mathematics

                    2 Numerics

                    3 Statistics

                    Goal Study Interrelationships

                    Detailed Look at PCA

                    Three Important (amp Interesting) Viewpoints

                    1 Mathematics

                    2 Numerics

                    3 Statistics

                    1st Review Linear Alg and Multivar Prob

                    Review of Linear Algebra

                    Vector Space

                    bull set of ldquovectorsrdquo

                    bull and ldquoscalarsrdquo (coefficients)

                    xa

                    Review of Linear Algebra

                    Vector Space

                    bull set of ldquovectorsrdquo

                    bull and ldquoscalarsrdquo (coefficients)

                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                    ( in space)

                    xa

                    i

                    ii xa

                    Review of Linear Algebra

                    Vector Space

                    bull set of ldquovectorsrdquo

                    bull and ldquoscalarsrdquo (coefficients)

                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                    ( in space)

                    eg

                    ldquo dim Euclidrsquon spacerdquo

                    xa

                    i

                    ii xa

                    d

                    d

                    d xx

                    x

                    x

                    x 1

                    1

                    d

                    Review of Linear Algebra (Cont)

                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                    Review of Linear Algebra (Cont)

                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                    Review of Linear Algebra (Cont)

                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                    Note Planes not Through the Origin

                    are not Subspaces

                    (Do not Contain )00 x

                    Review of Linear Algebra (Cont)

                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                    (all Linear Combos of them =

                    = Containing Hyperplane

                    through Origin)

                    Review of Linear Algebra (Cont)

                    Basis of Subspace Set of Vectors that

                    bull Span ie Everything is a Lin Com of them

                    bull are Linearly Indeprsquot ie Lin Com is Unique

                    Review of Linear Algebra (Cont)

                    Basis of Subspace Set of Vectors that

                    bull Span ie Everything is a Lin Com of them

                    bull are Linearly Indeprsquot ie Lin Com is Unique

                    bull eg ldquoUnit Vector Basisrdquo

                    bull Since

                    d

                    1

                    0

                    0

                    0

                    1

                    0

                    0

                    0

                    1

                    1

                    0

                    0

                    0

                    1

                    0

                    0

                    0

                    1

                    212

                    1

                    d

                    d

                    xxx

                    x

                    x

                    x

                    Review of Linear Algebra (Cont)

                    Basis Matrix of subspace of

                    Given a basis

                    create matrix of columns

                    dnvv 1

                    nddnd

                    n

                    n

                    vv

                    vv

                    vvB

                    1

                    111

                    1

                    Review of Linear Algebra (Cont)

                    Then linear combo is a matrix multiplicatrsquon

                    where

                    n

                    iii aBva

                    1

                    na

                    a

                    a 1

                    Review of Linear Algebra (Cont)

                    Then linear combo is a matrix multiplicatrsquon

                    where

                    Note Right Multiplication Gives

                    Linear Combination of Column Vectors

                    n

                    iii aBva

                    1

                    na

                    a

                    a 1

                    Review of Linear Algebra (Cont)

                    Then linear combo is a matrix multiplicatrsquon

                    where

                    Check sizes

                    n

                    iii aBva

                    1

                    na

                    a

                    a 1

                    )1()(1 nndd

                    Review of Linear Algebra (Cont)

                    Aside on Matrix Multiplication (linear transformatrsquon)

                    For matrices

                    Define the Matrix Product

                    mkk

                    m

                    aa

                    aa

                    A

                    1

                    111

                    nmm

                    n

                    bb

                    bb

                    B

                    1

                    111

                    m

                    iniik

                    m

                    iiik

                    m

                    inii

                    m

                    iii

                    baba

                    baba

                    AB

                    1

                    11

                    11

                    111

                    Review of Linear Algebra (Cont)

                    Aside on Matrix Multiplication (linear transformatrsquon)

                    For matrices

                    Define the Matrix Product

                    (Inner Products of Rows With Columns )

                    mkk

                    m

                    aa

                    aa

                    A

                    1

                    111

                    nmm

                    n

                    bb

                    bb

                    B

                    1

                    111

                    m

                    iniik

                    m

                    iiik

                    m

                    inii

                    m

                    iii

                    baba

                    baba

                    AB

                    1

                    11

                    11

                    111

                    A B

                    Review of Linear Algebra (Cont)

                    Aside on Matrix Multiplication (linear transformatrsquon)

                    For matrices

                    Define the Matrix Product

                    (Inner Products of Rows With Columns )

                    (Composition of Linear Transformations)

                    mkk

                    m

                    aa

                    aa

                    A

                    1

                    111

                    nmm

                    n

                    bb

                    bb

                    B

                    1

                    111

                    m

                    iniik

                    m

                    iiik

                    m

                    inii

                    m

                    iii

                    baba

                    baba

                    AB

                    1

                    11

                    11

                    111

                    A B

                    Review of Linear Algebra (Cont)

                    Aside on Matrix Multiplication (linear transformatrsquon)

                    For matrices

                    Define the Matrix Product

                    (Inner Products of Rows With Columns )

                    (Composition of Linear Transformations)

                    Often Useful to Check Sizes

                    mkk

                    m

                    aa

                    aa

                    A

                    1

                    111

                    nmm

                    n

                    bb

                    bb

                    B

                    1

                    111

                    m

                    iniik

                    m

                    iiik

                    m

                    inii

                    m

                    iii

                    baba

                    baba

                    AB

                    1

                    11

                    11

                    111

                    nmmknk

                    A B

                    Review of Linear Algebra (Cont)

                    Aside on Matrix Multiplication (linear transformatrsquon)

                    For matrices

                    Define the Matrix Product

                    (Inner Products of Rows With Columns )

                    (Composition of Linear Transformations)

                    Often Useful to Check Sizes

                    mkk

                    m

                    aa

                    aa

                    A

                    1

                    111

                    nmm

                    n

                    bb

                    bb

                    B

                    1

                    111

                    m

                    iniik

                    m

                    iiik

                    m

                    inii

                    m

                    iii

                    baba

                    baba

                    AB

                    1

                    11

                    11

                    111

                    nmmknk

                    A B

                    Review of Linear Algebra (Cont)

                    Matrix Trace

                    bull For a Square Matrix

                    bull Define

                    mmm

                    m

                    aa

                    aa

                    A

                    1

                    111

                    m

                    iiiaAtr

                    1)(

                    Review of Linear Algebra (Cont)

                    Matrix Trace

                    bull For a Square Matrix

                    bull Define

                    bull Trace Commutes with Matrix Multiplication

                    mmm

                    m

                    aa

                    aa

                    A

                    1

                    111

                    m

                    iiiaAtr

                    1)(

                    BAtrABtr

                    Review of Linear Algebra (Cont)

                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                    bull Number of Elements in a Basis (Unique)

                    Review of Linear Algebra (Cont)

                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                    bull Number of Elements in a Basis (Unique)

                    bull (Use Basis Above)

                    bull eg dim of a line is 1

                    bull eg dim of a plane is 2

                    dd dim

                    Review of Linear Algebra (Cont)

                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                    bull Number of Elements in a Basis (Unique)

                    bull (Use Basis Above)

                    bull eg dim of a line is 1

                    bull eg dim of a plane is 2

                    bull Dimension is ldquoDegrees of Freedomrdquo

                    (in Statistical Uses eg ANOVA)

                    dd dim

                    Review of Linear Algebra (Cont)

                    Norm of a Vector

                    bull in d 21

                    21

                    1

                    2 xxxx td

                    jj

                    Review of Linear Algebra (Cont)

                    Norm of a Vector

                    bull in

                    bull Idea length of the vector

                    d 2121

                    1

                    2 xxxx td

                    jj

                    Review of Linear Algebra (Cont)

                    Norm of a Vector

                    bull in

                    bull Idea length of the vector

                    bull Note strange properties for high

                    eg ldquolength of diagonal of unit cuberdquo =

                    d 2121

                    1

                    2 xxxx td

                    jj

                    d

                    d

                    Review of Linear Algebra (Cont)

                    Norm of a Vector (cont)

                    bull Length Normalized Vector

                    (has Length 1 thus on Surf of Unit Sphere

                    amp is a Direction Vector)

                    x

                    x

                    Review of Linear Algebra (Cont)

                    Norm of a Vector (cont)

                    bull Length Normalized Vector

                    (has Length 1 thus on Surf of Unit Sphere

                    amp is a Direction Vector)

                    bull Define Distance as

                    x

                    x

                    yxyxyxyxd t

                    Review of Linear Algebra (Cont)

                    Inner (Dot Scalar) Product

                    bull for Vectors and

                    yxyxyx td

                    jjj

                    1

                    x y

                    Review of Linear Algebra (Cont)

                    Inner (Dot Scalar) Product

                    bull for Vectors and

                    bull Related to Norm via

                    yxyxyx td

                    jjj

                    1

                    x y

                    21 xxx

                    Review of Linear Algebra (Cont)

                    Inner (Dot Scalar) Product (cont)

                    bull measures ldquoangle between and rdquo as

                    yyxx

                    yx

                    yx

                    yxyxangle

                    tt

                    t

                    11 cos

                    cos

                    x y

                    Review of Linear Algebra (Cont)

                    Inner (Dot Scalar) Product (cont)

                    bull measures ldquoangle between and rdquo as

                    bull key to Orthogonality ie Perpendiculrsquoty

                    if and only if

                    yyxx

                    yx

                    yx

                    yxyxangle

                    tt

                    t

                    11 cos

                    cos

                    x y

                    yx 0 yx

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis

                    bull All Orthogonal to each other

                    ie for

                    bull All have Length 1

                    ie for

                    nvv 1

                    1 ii vv

                    0 ii vv ii

                    ni 1

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis (cont)

                    bull Spectral Representation

                    where

                    nvv 1

                    n

                    iii vax

                    1

                    ii vxa

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis (cont)

                    bull Spectral Representation

                    where

                    (Coefficient is Inner Product Cool Notation)

                    nvv 1

                    n

                    iii vax

                    1

                    ii vxa

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis (cont)

                    bull Spectral Representation

                    where

                    Check

                    nvv 1

                    n

                    iii vax

                    1

                    ii vxa

                    iii

                    n

                    iii

                    n

                    iiii avvavvavx

                    1

                    1

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis (cont)

                    bull Spectral Representation

                    where

                    Check

                    bull Matrix Notation where ie

                    For the Basis Matrix

                    nvv 1

                    n

                    iii vax

                    1

                    ii vxa

                    iii

                    n

                    iii

                    n

                    iiii avvavvavx

                    1

                    1

                    aBx Bxa tt xBa t

                    nvvB 1

                    Review of Linear Algebra (Cont)

                    Orthonormal Basis (cont)

                    bull Spectral Representation

                    where

                    Check

                    bull Matrix Notation where ie

                    is called transform of

                    (eg Fourier or Wavelet)

                    nvv 1

                    n

                    iii vax

                    1

                    ii vxa

                    iii

                    n

                    iii

                    n

                    iiii avvavvavx

                    1

                    1

                    aBx Bxa tt xBa t

                    xa

                    Review of Linear Algebra (Cont)

                    Parseval identity for

                    in subsp genrsquod by o n basis

                    x

                    nvv 1

                    2

                    1

                    22

                    1

                    2 aavxx

                    n

                    ii

                    n

                    ii

                    Review of Linear Algebra (Cont)

                    Parseval identity for

                    in subsp genrsquod by o n basis

                    bull Pythagorean theorem

                    bull ldquoDecomposition of Energyrdquo

                    bull ANOVA - sums of squares

                    x

                    nvv 1

                    2

                    1

                    22

                    1

                    2 aavxx

                    n

                    ii

                    n

                    ii

                    Review of Linear Algebra (Cont)

                    Parseval identity for

                    in subsp genrsquod by o n basis

                    bull Pythagorean theorem

                    bull ldquoDecomposition of Energyrdquo

                    bull ANOVA - sums of squares

                    bull Transform has same length as

                    ie ldquorotation in rdquo

                    x

                    nvv 1

                    2

                    1

                    22

                    1

                    2 aavxx

                    n

                    ii

                    n

                    ii

                    a xd

                    Projection of a Vector onto a Subspace

                    bull Idea Member of that is Closest to

                    (ie ldquoBest Approxrsquonrdquo)

                    Review of Linear Algebra (Cont)x

                    xV

                    V

                    Projection of a Vector onto a Subspace

                    bull Idea Member of that is Closest to

                    (ie ldquoBest Approxrsquonrdquo)

                    bull Find that Solves

                    (ldquoLeast Squaresrdquo)

                    Review of Linear Algebra (Cont)x

                    xV

                    V

                    VxPV vxVv

                    min

                    Projection of a Vector onto a Subspace

                    bull Idea Member of that is Closest to

                    (ie ldquoBest Approxrsquonrdquo)

                    bull Find that Solves

                    (ldquoLeast Squaresrdquo)

                    bull For Inner Product (Hilbert) Space

                    Exists and is Unique

                    Review of Linear Algebra (Cont)x

                    xV

                    V

                    VxPV vxVv

                    min

                    xPV

                    Projection of a Vector onto a Subspace (cont)

                    bull General Solution in for Basis Matrix

                    Review of Linear Algebra (Cont)

                    d VB

                    xBBBBxP tVV

                    tVVV

                    1

                    Projection of a Vector onto a Subspace (cont)

                    bull General Solution in for Basis Matrix

                    bull So Projrsquon Operator is Matrix Multrsquon

                    (thus projection is another linear operation)

                    Review of Linear Algebra (Cont)

                    d VB

                    xBBBBxP tVV

                    tVVV

                    1

                    tVV

                    tVVV BBBBP

                    1

                    Projection of a Vector onto a Subspace (cont)

                    bull General Solution in for Basis Matrix

                    bull So Projrsquon Operator is Matrix Multrsquon

                    (thus projection is another linear operation)

                    (note same operation underlies least squares)

                    Review of Linear Algebra (Cont)

                    d VB

                    xBBBBxP tVV

                    tVVV

                    1

                    tVV

                    tVVV BBBBP

                    1

                    Review of Linear Algebra (Cont)

                    Projection using Orthonormal Basis

                    bull Basis Matrix is Orthonormal

                    nnVtV IBB

                    10

                    01

                    1

                    111

                    1

                    1

                    nnn

                    n

                    ntn

                    t

                    vvvv

                    vvvv

                    vv

                    v

                    v

                    nvv 1

                    Review of Linear Algebra (Cont)

                    Projection using Orthonormal Basis

                    bull Basis Matrix is Orthonormal

                    bull So =

                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                    (Recall Right Multrsquon)

                    nnVtV IBB

                    10

                    01

                    1

                    111

                    1

                    1

                    nnn

                    n

                    ntn

                    t

                    vvvv

                    vvvv

                    vv

                    v

                    v

                    xBBxP tVVV

                    x V

                    nvv 1

                    Review of Linear Algebra (Cont)

                    Projection using Orthonormal Basis (cont)

                    bull For Orthogonal Complement

                    and

                    V

                    xPxPx VV 222xPxPx VV

                    Review of Linear Algebra (Cont)

                    Projection using Orthonormal Basis (cont)

                    bull For Orthogonal Complement

                    and

                    bull Parseval Inequality

                    V

                    xPxPx VV 222xPxPx VV

                    2

                    1

                    22

                    1

                    22 aavxxxP

                    n

                    ii

                    n

                    iiV

                    Review of Linear Algebra (Cont)

                    (Real) Unitary Matrices with

                    bull Orthonormal Basis Matrix

                    (So All of Above Applies)

                    ddU IUU t

                    Review of Linear Algebra (Cont)

                    (Real) Unitary Matrices with

                    bull Orthonormal Basis Matrix

                    (So All of Above Applies)

                    bull Note Transformrsquon is Distance Preserving

                    ddU IUU t

                    yxdyxyxyxUyUxUdn

                    i ii 2

                    1

                    Review of Linear Algebra (Cont)

                    (Real) Unitary Matrices with

                    bull Orthonormal Basis Matrix

                    (So All of Above Applies)

                    bull Note Transformrsquon is Distance Preserving

                    bull Lin Trans (Mult by ) is ~ Rotation

                    bull But also Includes ldquoMirror Imagesrdquo

                    ddU IUU t

                    yxdyxyxyxUyUxUdn

                    i ii 2

                    1

                    U

                    Review of Linear Algebra (Cont)

                    Singular Value Decomposition (SVD)

                    For a Matrix

                    Find

                    ndX

                    Review of Linear Algebra (Cont)

                    Singular Value Decomposition (SVD)

                    For a Matrix

                    Find a Diagonal Matrix

                    with Entries

                    called Singular Values

                    ndX

                    ndS

                    )min(1 ndss

                    Review of Linear Algebra (Cont)

                    Singular Value Decomposition (SVD)

                    For a Matrix

                    Find a Diagonal Matrix

                    with Entries

                    called Singular Values

                    And Unitary (Rotation) Matrices

                    (recall )

                    ndX

                    ndS

                    )min(1 ndss

                    ddU nnV

                    IVVUU tt

                    Review of Linear Algebra (Cont)

                    Singular Value Decomposition (SVD)

                    For a Matrix

                    Find a Diagonal Matrix

                    with Entries

                    called Singular Values

                    And Unitary (Rotation) Matrices

                    (recall )

                    So That

                    ndX

                    ndS

                    )min(1 ndss

                    ddU nnV

                    IVVUU tt tUSVX

                    Review of Linear Algebra (Cont)

                    Intuition behind Singular Value Decomposition

                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                    vVSUvVSUvX tt

                    Review of Linear Algebra (Cont)

                    Intuition behind Singular Value Decomposition

                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                    bull First rotate

                    X

                    vVSUvVSUvX tt

                    Review of Linear Algebra (Cont)

                    Intuition behind Singular Value Decomposition

                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                    bull First rotate

                    bull Second rescale coordinate axes (by )

                    X

                    vVSUvVSUvX tt

                    is

                    Review of Linear Algebra (Cont)

                    Intuition behind Singular Value Decomposition

                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                    bull First rotate

                    bull Second rescale coordinate axes (by )

                    bull Third rotate again

                    X

                    vVSUvVSUvX tt

                    is

                    Review of Linear Algebra (Cont)

                    Intuition behind Singular Value Decomposition

                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                    bull First rotate

                    bull Second rescale coordinate axes (by )

                    bull Third rotate again

                    bull ie have diagonalized the transformation

                    X

                    vVSUvVSUvX tt

                    is

                    SVD Compact Representation

                    Useful Labeling

                    Singular Values in Increasing Order

                    Review of Linear Algebra (Cont)

                    )min(1 dnss

                    SVD Compact Representation

                    Useful Labeling

                    Singular Values in Increasing Order

                    Note singular values = 0 can be omitted

                    (Since do ldquo0-Stretchingrdquo)

                    Review of Linear Algebra (Cont)

                    )min(1 dnss

                    Review of Linear Algebra (Cont)

                    r

                    SVD Compact Representation

                    Useful Labeling

                    Singular Values in Increasing Order

                    Note singular values = 0 can be omitted

                    Let = of positive singular values

                    )min(1 dnss

                    Review of Linear Algebra (Cont)

                    r

                    SVD Compact Representation

                    Useful Labeling

                    Singular Values in Increasing Order

                    Note singular values = 0 can be omitted

                    Let = of positive singular values

                    Then

                    Where are truncations of

                    trnrrrd VSUX

                    VSU

                    )min(1 dnss

                    Review of Linear Algebra (Cont)

                    SVD Full Representation

                    =

                    Graphics Display Assumes

                    ndX ddU ndS nn

                    tV

                    nd

                    Review of Linear Algebra (Cont)

                    SVD Full Representation

                    =

                    Full Rank Basis Matrix

                    ndX ddU ndS nn

                    tV

                    Review of Linear Algebra (Cont)

                    SVD Full Representation

                    =

                    Full Rank Basis Matrix

                    All 0s in Bottom

                    ndX ddU ndS nn

                    tV

                    Review of Linear Algebra (Cont)

                    SVD Reduced Representation

                    =

                    These Columns Get 0ed Out

                    ndX ddU nnS nn

                    tV

                    nnd 0

                    Review of Linear Algebra (Cont)

                    SVD Reduced Representation

                    =ndX ndU nnS nn

                    tV

                    Review of Linear Algebra (Cont)

                    SVD Reduced Representation

                    =

                    Also Some of These May be 0

                    ndX ndU nnS nn

                    tV

                    Review of Linear Algebra (Cont)

                    SVD Compact Representation

                    = ndX rdU

                    rrS nrtV

                    0

                    Review of Linear Algebra (Cont)

                    SVD Compact Representation

                    =

                    These Get 0ed Out

                    ndX rdU

                    rrS nrtV

                    0

                    Review of Linear Algebra (Cont)

                    SVD Compact Representation

                    = ndX rdU

                    rrS nrtV

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition

                    For a (Symmetric) Square Matrix

                    Find

                    ddX

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition

                    For a (Symmetric) Square Matrix

                    Find a Diagonal Matrix

                    Called Eigenvalues

                    Convenient Ordering

                    ddX

                    d

                    D

                    0

                    01

                    n 1

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition

                    For a (Symmetric) Square Matrix

                    Find a Diagonal Matrix

                    And an Orthonormal Matrix

                    (ie )

                    ddX

                    d

                    D

                    0

                    01

                    ddB

                    ddtt IBBBB

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition

                    For a (Symmetric) Square Matrix

                    Find a Diagonal Matrix

                    And an Orthonormal Matrix

                    (ie )

                    So that ie

                    ddX

                    d

                    D

                    0

                    01

                    ddB

                    ddtt IBBBB

                    DBBX tBDBX

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                    (looks similar)

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                    Complex (uses )

                    VU

                    1i

                    Review of Linear Algebra (Cont)

                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                    VU

                    X

                    BVU

                    1i

                    • Participant Presentations
                    • PCA to find clusters
                    • Statistical Smoothing
                    • Kernel Density Estimation
                    • Scatterplot Smoothing
                    • Statistical Smoothing (2)
                    • SiZer Background
                    • SiZer Background (2)
                    • SiZer Background (3)
                    • SiZer Background (4)
                    • SiZer Background (5)
                    • SiZer Overview
                    • PCA to find clusters (2)
                    • PCA to find clusters (3)
                    • PCA to find clusters (4)
                    • PCA to find clusters (5)
                    • PCA to find clusters (6)
                    • PCA to find clusters (7)
                    • Recall Yeast Cell Cycle Data
                    • Yeast Cell Cycle Data FDA View
                    • Yeast Cell Cycle Data FDA View (2)
                    • Yeast Cell Cycle Data FDA View (3)
                    • Yeast Cell Cycles Freq 2 Proj
                    • Frequency 2 Analysis
                    • Frequency 2 Analysis (2)
                    • Frequency 2 Analysis (3)
                    • Yeast Cell Cycle
                    • SiZer Study of Distrsquon of Angles
                    • Reclassification of Major Genes
                    • Compare to Previous Classifrsquon
                    • New Subpopulation View
                    • New Subpopulation View (2)
                    • Detailed Look at PCA
                    • PCA Rediscovery ndash Renaming
                    • PCA Rediscovery ndash Renaming (2)
                    • PCA Rediscovery ndash Renaming (3)
                    • PCA Rediscovery ndash Renaming (4)
                    • PCA Rediscovery ndash Renaming (5)
                    • An Interesting Historical Note
                    • An Interesting Historical Note (2)
                    • Detailed Look at PCA (2)
                    • Detailed Look at PCA (3)
                    • Review of Linear Algebra
                    • Review of Linear Algebra (2)
                    • Review of Linear Algebra (3)
                    • Review of Linear Algebra (Cont)
                    • Review of Linear Algebra (Cont) (2)
                    • Review of Linear Algebra (Cont) (3)
                    • Review of Linear Algebra (Cont) (4)
                    • Review of Linear Algebra (Cont) (5)
                    • Review of Linear Algebra (Cont) (6)
                    • Review of Linear Algebra (Cont) (7)
                    • Review of Linear Algebra (Cont) (8)
                    • Review of Linear Algebra (Cont) (9)
                    • Review of Linear Algebra (Cont) (10)
                    • Review of Linear Algebra (Cont) (11)
                    • Review of Linear Algebra (Cont) (12)
                    • Review of Linear Algebra (Cont) (13)
                    • Review of Linear Algebra (Cont) (14)
                    • Review of Linear Algebra (Cont) (15)
                    • Review of Linear Algebra (Cont) (16)
                    • Review of Linear Algebra (Cont) (17)
                    • Review of Linear Algebra (Cont) (18)
                    • Review of Linear Algebra (Cont) (19)
                    • Review of Linear Algebra (Cont) (20)
                    • Review of Linear Algebra (Cont) (21)
                    • Review of Linear Algebra (Cont) (22)
                    • Review of Linear Algebra (Cont) (23)
                    • Review of Linear Algebra (Cont) (24)
                    • Review of Linear Algebra (Cont) (25)
                    • Review of Linear Algebra (Cont) (26)
                    • Review of Linear Algebra (Cont) (27)
                    • Review of Linear Algebra (Cont) (28)
                    • Review of Linear Algebra (Cont) (29)
                    • Review of Linear Algebra (Cont) (30)
                    • Review of Linear Algebra (Cont) (31)
                    • Review of Linear Algebra (Cont) (32)
                    • Review of Linear Algebra (Cont) (33)
                    • Review of Linear Algebra (Cont) (34)
                    • Review of Linear Algebra (Cont) (35)
                    • Review of Linear Algebra (Cont) (36)
                    • Review of Linear Algebra (Cont) (37)
                    • Review of Linear Algebra (Cont) (38)
                    • Review of Linear Algebra (Cont) (39)
                    • Review of Linear Algebra (Cont) (40)
                    • Review of Linear Algebra (Cont) (41)
                    • Review of Linear Algebra (Cont) (42)
                    • Review of Linear Algebra (Cont) (43)
                    • Review of Linear Algebra (Cont) (44)
                    • Review of Linear Algebra (Cont) (45)
                    • Review of Linear Algebra (Cont) (46)
                    • Review of Linear Algebra (Cont) (47)
                    • Review of Linear Algebra (Cont) (48)
                    • Review of Linear Algebra (Cont) (49)
                    • Review of Linear Algebra (Cont) (50)
                    • Review of Linear Algebra (Cont) (51)
                    • Review of Linear Algebra (Cont) (52)
                    • Review of Linear Algebra (Cont) (53)
                    • Review of Linear Algebra (Cont) (54)
                    • Review of Linear Algebra (Cont) (55)
                    • Review of Linear Algebra (Cont) (56)
                    • Review of Linear Algebra (Cont) (57)
                    • Review of Linear Algebra (Cont) (58)
                    • Review of Linear Algebra (Cont) (59)
                    • Review of Linear Algebra (Cont) (60)
                    • Review of Linear Algebra (Cont) (61)
                    • Review of Linear Algebra (Cont) (62)
                    • Review of Linear Algebra (Cont) (63)
                    • Review of Linear Algebra (Cont) (64)
                    • Review of Linear Algebra (Cont) (65)
                    • Review of Linear Algebra (Cont) (66)
                    • Review of Linear Algebra (Cont) (67)
                    • Review of Linear Algebra (Cont) (68)
                    • Review of Linear Algebra (Cont) (69)
                    • Review of Linear Algebra (Cont) (70)
                    • Review of Linear Algebra (Cont) (71)
                    • Review of Linear Algebra (Cont) (72)
                    • Review of Linear Algebra (Cont) (73)
                    • Review of Linear Algebra (Cont) (74)
                    • Review of Linear Algebra (Cont) (75)
                    • Review of Linear Algebra (Cont) (76)
                    • Review of Linear Algebra (Cont) (77)
                    • Review of Linear Algebra (Cont) (78)
                    • Review of Linear Algebra (Cont) (79)
                    • Review of Linear Algebra (Cont) (80)
                    • Review of Linear Algebra (Cont) (81)

                      SiZer BackgroundRecall

                      Hidalgo

                      Stamps

                      Data

                      >

                      SiZer Overview

                      Would you like to try smoothing amp

                      SiZer

                      bull Marron Software Website as Before

                      bull In ldquoSmoothingrdquo Directory

                      ndash kdeSMm

                      ndash nprSMm

                      ndash sizerSMm

                      bull Recall ldquogtgt help sizerSMrdquo for

                      usage

                      PCA to find clustersReturn to PCA of Mass Flux Data

                      PCA to find clustersSiZer analysis of Mass Flux PC1

                      PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                      PCA to find clustersSiZer analysis of Mass Flux PC1

                      Also in Curvature

                      PCA to find clustersSiZer analysis of Mass Flux PC1

                      And in Other Comprsquos

                      PCA to find clustersSiZer analysis of Mass Flux PC1

                      Conclusion

                      bull Found 3 significant clusters

                      bull Correspond to 3 known ldquocloud typesrdquo

                      bull Worth deeper investigation

                      Recall Yeast Cell Cycle Data

                      bull ldquoGene Expressionrdquo ndash Micro-array data

                      bull Data (after major preprocessing) Expression ldquolevelrdquo of

                      bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                      10s)bull Interesting statistical issue

                      High Dimension Low Sample Size data

                      (HDLSS)

                      Yeast Cell Cycle Data FDA View

                      Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                      Yeast Cell Cycle Data FDA View

                      Periodic genes

                      Naiumlve

                      approach

                      Simple PCA

                      Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                      ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                      ldquovariationrdquobull PCA finds ldquodirections of maximal

                      variationrdquobull Often but not always same as

                      ldquointeresting directionsrdquobull Here need better approach to study

                      periodicities

                      Yeast Cell Cycles Freq 2 Proj

                      PCA on

                      Freq 2

                      Periodic

                      Component

                      Of Data

                      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                      Approach from Zhao Marron amp Wells (2004)

                      Frequency 2 Analysis

                      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                      classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                      notable differencesbull Now try to improve ldquophase classificationrdquo

                      Yeast Cell CycleRevisit ldquophase classificationrdquo

                      approachbull Use outer 200 genes

                      (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                      (finds significant bumps etc in histogram)

                      bull Carefully redrew boundariesbull Check by studying kde angles

                      SiZer Study of Distrsquon of Angles

                      Reclassification of Major Genes

                      Compare to Previous Classifrsquon

                      New Subpopulation View

                      New Subpopulation View

                      NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                      Detailed Look at PCA

                      Now Study ldquoFolklorerdquo More Carefully

                      bull BackGround

                      bull History

                      bull Underpinnings

                      (Mathematical amp Computational)

                      Good Overall Reference Jolliffe (2002)

                      PCA Rediscovery ndash Renaming

                      Statistics Principal Component Analysis (PCA)

                      PCA Rediscovery ndash Renaming

                      Statistics Principal Component Analysis (PCA)

                      Social Sciences Factor Analysis (PCA is a subset)

                      PCA Rediscovery ndash Renaming

                      Statistics Principal Component Analysis (PCA)

                      Social Sciences Factor Analysis (PCA is a subset)

                      Probability Electrical EngKarhunen ndash Loeve expansion

                      PCA Rediscovery ndash Renaming

                      Statistics Principal Component Analysis (PCA)

                      Social Sciences Factor Analysis (PCA is a subset)

                      Probability Electrical EngKarhunen ndash Loeve expansion

                      Applied MathematicsProper Orthogonal Decomposition (POD)

                      PCA Rediscovery ndash Renaming

                      Statistics Principal Component Analysis (PCA)

                      Social Sciences Factor Analysis (PCA is a subset)

                      Probability Electrical EngKarhunen ndash Loeve expansion

                      Applied MathematicsProper Orthogonal Decomposition (POD)

                      Geo-Sciences Empirical Orthogonal Functions (EOF)

                      An Interesting Historical Note

                      The 1st () application of PCA to Functional

                      Data Analysis

                      An Interesting Historical Note

                      The 1st () application of PCA to Functional

                      Data Analysis

                      Rao (1958)

                      1st Paper with ldquoCurves as Data Objectsrdquo

                      viewpoint

                      Detailed Look at PCA

                      Three Important (amp Interesting) Viewpoints

                      1 Mathematics

                      2 Numerics

                      3 Statistics

                      Goal Study Interrelationships

                      Detailed Look at PCA

                      Three Important (amp Interesting) Viewpoints

                      1 Mathematics

                      2 Numerics

                      3 Statistics

                      1st Review Linear Alg and Multivar Prob

                      Review of Linear Algebra

                      Vector Space

                      bull set of ldquovectorsrdquo

                      bull and ldquoscalarsrdquo (coefficients)

                      xa

                      Review of Linear Algebra

                      Vector Space

                      bull set of ldquovectorsrdquo

                      bull and ldquoscalarsrdquo (coefficients)

                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                      ( in space)

                      xa

                      i

                      ii xa

                      Review of Linear Algebra

                      Vector Space

                      bull set of ldquovectorsrdquo

                      bull and ldquoscalarsrdquo (coefficients)

                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                      ( in space)

                      eg

                      ldquo dim Euclidrsquon spacerdquo

                      xa

                      i

                      ii xa

                      d

                      d

                      d xx

                      x

                      x

                      x 1

                      1

                      d

                      Review of Linear Algebra (Cont)

                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                      Review of Linear Algebra (Cont)

                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                      Review of Linear Algebra (Cont)

                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                      Note Planes not Through the Origin

                      are not Subspaces

                      (Do not Contain )00 x

                      Review of Linear Algebra (Cont)

                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                      (all Linear Combos of them =

                      = Containing Hyperplane

                      through Origin)

                      Review of Linear Algebra (Cont)

                      Basis of Subspace Set of Vectors that

                      bull Span ie Everything is a Lin Com of them

                      bull are Linearly Indeprsquot ie Lin Com is Unique

                      Review of Linear Algebra (Cont)

                      Basis of Subspace Set of Vectors that

                      bull Span ie Everything is a Lin Com of them

                      bull are Linearly Indeprsquot ie Lin Com is Unique

                      bull eg ldquoUnit Vector Basisrdquo

                      bull Since

                      d

                      1

                      0

                      0

                      0

                      1

                      0

                      0

                      0

                      1

                      1

                      0

                      0

                      0

                      1

                      0

                      0

                      0

                      1

                      212

                      1

                      d

                      d

                      xxx

                      x

                      x

                      x

                      Review of Linear Algebra (Cont)

                      Basis Matrix of subspace of

                      Given a basis

                      create matrix of columns

                      dnvv 1

                      nddnd

                      n

                      n

                      vv

                      vv

                      vvB

                      1

                      111

                      1

                      Review of Linear Algebra (Cont)

                      Then linear combo is a matrix multiplicatrsquon

                      where

                      n

                      iii aBva

                      1

                      na

                      a

                      a 1

                      Review of Linear Algebra (Cont)

                      Then linear combo is a matrix multiplicatrsquon

                      where

                      Note Right Multiplication Gives

                      Linear Combination of Column Vectors

                      n

                      iii aBva

                      1

                      na

                      a

                      a 1

                      Review of Linear Algebra (Cont)

                      Then linear combo is a matrix multiplicatrsquon

                      where

                      Check sizes

                      n

                      iii aBva

                      1

                      na

                      a

                      a 1

                      )1()(1 nndd

                      Review of Linear Algebra (Cont)

                      Aside on Matrix Multiplication (linear transformatrsquon)

                      For matrices

                      Define the Matrix Product

                      mkk

                      m

                      aa

                      aa

                      A

                      1

                      111

                      nmm

                      n

                      bb

                      bb

                      B

                      1

                      111

                      m

                      iniik

                      m

                      iiik

                      m

                      inii

                      m

                      iii

                      baba

                      baba

                      AB

                      1

                      11

                      11

                      111

                      Review of Linear Algebra (Cont)

                      Aside on Matrix Multiplication (linear transformatrsquon)

                      For matrices

                      Define the Matrix Product

                      (Inner Products of Rows With Columns )

                      mkk

                      m

                      aa

                      aa

                      A

                      1

                      111

                      nmm

                      n

                      bb

                      bb

                      B

                      1

                      111

                      m

                      iniik

                      m

                      iiik

                      m

                      inii

                      m

                      iii

                      baba

                      baba

                      AB

                      1

                      11

                      11

                      111

                      A B

                      Review of Linear Algebra (Cont)

                      Aside on Matrix Multiplication (linear transformatrsquon)

                      For matrices

                      Define the Matrix Product

                      (Inner Products of Rows With Columns )

                      (Composition of Linear Transformations)

                      mkk

                      m

                      aa

                      aa

                      A

                      1

                      111

                      nmm

                      n

                      bb

                      bb

                      B

                      1

                      111

                      m

                      iniik

                      m

                      iiik

                      m

                      inii

                      m

                      iii

                      baba

                      baba

                      AB

                      1

                      11

                      11

                      111

                      A B

                      Review of Linear Algebra (Cont)

                      Aside on Matrix Multiplication (linear transformatrsquon)

                      For matrices

                      Define the Matrix Product

                      (Inner Products of Rows With Columns )

                      (Composition of Linear Transformations)

                      Often Useful to Check Sizes

                      mkk

                      m

                      aa

                      aa

                      A

                      1

                      111

                      nmm

                      n

                      bb

                      bb

                      B

                      1

                      111

                      m

                      iniik

                      m

                      iiik

                      m

                      inii

                      m

                      iii

                      baba

                      baba

                      AB

                      1

                      11

                      11

                      111

                      nmmknk

                      A B

                      Review of Linear Algebra (Cont)

                      Aside on Matrix Multiplication (linear transformatrsquon)

                      For matrices

                      Define the Matrix Product

                      (Inner Products of Rows With Columns )

                      (Composition of Linear Transformations)

                      Often Useful to Check Sizes

                      mkk

                      m

                      aa

                      aa

                      A

                      1

                      111

                      nmm

                      n

                      bb

                      bb

                      B

                      1

                      111

                      m

                      iniik

                      m

                      iiik

                      m

                      inii

                      m

                      iii

                      baba

                      baba

                      AB

                      1

                      11

                      11

                      111

                      nmmknk

                      A B

                      Review of Linear Algebra (Cont)

                      Matrix Trace

                      bull For a Square Matrix

                      bull Define

                      mmm

                      m

                      aa

                      aa

                      A

                      1

                      111

                      m

                      iiiaAtr

                      1)(

                      Review of Linear Algebra (Cont)

                      Matrix Trace

                      bull For a Square Matrix

                      bull Define

                      bull Trace Commutes with Matrix Multiplication

                      mmm

                      m

                      aa

                      aa

                      A

                      1

                      111

                      m

                      iiiaAtr

                      1)(

                      BAtrABtr

                      Review of Linear Algebra (Cont)

                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                      bull Number of Elements in a Basis (Unique)

                      Review of Linear Algebra (Cont)

                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                      bull Number of Elements in a Basis (Unique)

                      bull (Use Basis Above)

                      bull eg dim of a line is 1

                      bull eg dim of a plane is 2

                      dd dim

                      Review of Linear Algebra (Cont)

                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                      bull Number of Elements in a Basis (Unique)

                      bull (Use Basis Above)

                      bull eg dim of a line is 1

                      bull eg dim of a plane is 2

                      bull Dimension is ldquoDegrees of Freedomrdquo

                      (in Statistical Uses eg ANOVA)

                      dd dim

                      Review of Linear Algebra (Cont)

                      Norm of a Vector

                      bull in d 21

                      21

                      1

                      2 xxxx td

                      jj

                      Review of Linear Algebra (Cont)

                      Norm of a Vector

                      bull in

                      bull Idea length of the vector

                      d 2121

                      1

                      2 xxxx td

                      jj

                      Review of Linear Algebra (Cont)

                      Norm of a Vector

                      bull in

                      bull Idea length of the vector

                      bull Note strange properties for high

                      eg ldquolength of diagonal of unit cuberdquo =

                      d 2121

                      1

                      2 xxxx td

                      jj

                      d

                      d

                      Review of Linear Algebra (Cont)

                      Norm of a Vector (cont)

                      bull Length Normalized Vector

                      (has Length 1 thus on Surf of Unit Sphere

                      amp is a Direction Vector)

                      x

                      x

                      Review of Linear Algebra (Cont)

                      Norm of a Vector (cont)

                      bull Length Normalized Vector

                      (has Length 1 thus on Surf of Unit Sphere

                      amp is a Direction Vector)

                      bull Define Distance as

                      x

                      x

                      yxyxyxyxd t

                      Review of Linear Algebra (Cont)

                      Inner (Dot Scalar) Product

                      bull for Vectors and

                      yxyxyx td

                      jjj

                      1

                      x y

                      Review of Linear Algebra (Cont)

                      Inner (Dot Scalar) Product

                      bull for Vectors and

                      bull Related to Norm via

                      yxyxyx td

                      jjj

                      1

                      x y

                      21 xxx

                      Review of Linear Algebra (Cont)

                      Inner (Dot Scalar) Product (cont)

                      bull measures ldquoangle between and rdquo as

                      yyxx

                      yx

                      yx

                      yxyxangle

                      tt

                      t

                      11 cos

                      cos

                      x y

                      Review of Linear Algebra (Cont)

                      Inner (Dot Scalar) Product (cont)

                      bull measures ldquoangle between and rdquo as

                      bull key to Orthogonality ie Perpendiculrsquoty

                      if and only if

                      yyxx

                      yx

                      yx

                      yxyxangle

                      tt

                      t

                      11 cos

                      cos

                      x y

                      yx 0 yx

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis

                      bull All Orthogonal to each other

                      ie for

                      bull All have Length 1

                      ie for

                      nvv 1

                      1 ii vv

                      0 ii vv ii

                      ni 1

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis (cont)

                      bull Spectral Representation

                      where

                      nvv 1

                      n

                      iii vax

                      1

                      ii vxa

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis (cont)

                      bull Spectral Representation

                      where

                      (Coefficient is Inner Product Cool Notation)

                      nvv 1

                      n

                      iii vax

                      1

                      ii vxa

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis (cont)

                      bull Spectral Representation

                      where

                      Check

                      nvv 1

                      n

                      iii vax

                      1

                      ii vxa

                      iii

                      n

                      iii

                      n

                      iiii avvavvavx

                      1

                      1

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis (cont)

                      bull Spectral Representation

                      where

                      Check

                      bull Matrix Notation where ie

                      For the Basis Matrix

                      nvv 1

                      n

                      iii vax

                      1

                      ii vxa

                      iii

                      n

                      iii

                      n

                      iiii avvavvavx

                      1

                      1

                      aBx Bxa tt xBa t

                      nvvB 1

                      Review of Linear Algebra (Cont)

                      Orthonormal Basis (cont)

                      bull Spectral Representation

                      where

                      Check

                      bull Matrix Notation where ie

                      is called transform of

                      (eg Fourier or Wavelet)

                      nvv 1

                      n

                      iii vax

                      1

                      ii vxa

                      iii

                      n

                      iii

                      n

                      iiii avvavvavx

                      1

                      1

                      aBx Bxa tt xBa t

                      xa

                      Review of Linear Algebra (Cont)

                      Parseval identity for

                      in subsp genrsquod by o n basis

                      x

                      nvv 1

                      2

                      1

                      22

                      1

                      2 aavxx

                      n

                      ii

                      n

                      ii

                      Review of Linear Algebra (Cont)

                      Parseval identity for

                      in subsp genrsquod by o n basis

                      bull Pythagorean theorem

                      bull ldquoDecomposition of Energyrdquo

                      bull ANOVA - sums of squares

                      x

                      nvv 1

                      2

                      1

                      22

                      1

                      2 aavxx

                      n

                      ii

                      n

                      ii

                      Review of Linear Algebra (Cont)

                      Parseval identity for

                      in subsp genrsquod by o n basis

                      bull Pythagorean theorem

                      bull ldquoDecomposition of Energyrdquo

                      bull ANOVA - sums of squares

                      bull Transform has same length as

                      ie ldquorotation in rdquo

                      x

                      nvv 1

                      2

                      1

                      22

                      1

                      2 aavxx

                      n

                      ii

                      n

                      ii

                      a xd

                      Projection of a Vector onto a Subspace

                      bull Idea Member of that is Closest to

                      (ie ldquoBest Approxrsquonrdquo)

                      Review of Linear Algebra (Cont)x

                      xV

                      V

                      Projection of a Vector onto a Subspace

                      bull Idea Member of that is Closest to

                      (ie ldquoBest Approxrsquonrdquo)

                      bull Find that Solves

                      (ldquoLeast Squaresrdquo)

                      Review of Linear Algebra (Cont)x

                      xV

                      V

                      VxPV vxVv

                      min

                      Projection of a Vector onto a Subspace

                      bull Idea Member of that is Closest to

                      (ie ldquoBest Approxrsquonrdquo)

                      bull Find that Solves

                      (ldquoLeast Squaresrdquo)

                      bull For Inner Product (Hilbert) Space

                      Exists and is Unique

                      Review of Linear Algebra (Cont)x

                      xV

                      V

                      VxPV vxVv

                      min

                      xPV

                      Projection of a Vector onto a Subspace (cont)

                      bull General Solution in for Basis Matrix

                      Review of Linear Algebra (Cont)

                      d VB

                      xBBBBxP tVV

                      tVVV

                      1

                      Projection of a Vector onto a Subspace (cont)

                      bull General Solution in for Basis Matrix

                      bull So Projrsquon Operator is Matrix Multrsquon

                      (thus projection is another linear operation)

                      Review of Linear Algebra (Cont)

                      d VB

                      xBBBBxP tVV

                      tVVV

                      1

                      tVV

                      tVVV BBBBP

                      1

                      Projection of a Vector onto a Subspace (cont)

                      bull General Solution in for Basis Matrix

                      bull So Projrsquon Operator is Matrix Multrsquon

                      (thus projection is another linear operation)

                      (note same operation underlies least squares)

                      Review of Linear Algebra (Cont)

                      d VB

                      xBBBBxP tVV

                      tVVV

                      1

                      tVV

                      tVVV BBBBP

                      1

                      Review of Linear Algebra (Cont)

                      Projection using Orthonormal Basis

                      bull Basis Matrix is Orthonormal

                      nnVtV IBB

                      10

                      01

                      1

                      111

                      1

                      1

                      nnn

                      n

                      ntn

                      t

                      vvvv

                      vvvv

                      vv

                      v

                      v

                      nvv 1

                      Review of Linear Algebra (Cont)

                      Projection using Orthonormal Basis

                      bull Basis Matrix is Orthonormal

                      bull So =

                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                      (Recall Right Multrsquon)

                      nnVtV IBB

                      10

                      01

                      1

                      111

                      1

                      1

                      nnn

                      n

                      ntn

                      t

                      vvvv

                      vvvv

                      vv

                      v

                      v

                      xBBxP tVVV

                      x V

                      nvv 1

                      Review of Linear Algebra (Cont)

                      Projection using Orthonormal Basis (cont)

                      bull For Orthogonal Complement

                      and

                      V

                      xPxPx VV 222xPxPx VV

                      Review of Linear Algebra (Cont)

                      Projection using Orthonormal Basis (cont)

                      bull For Orthogonal Complement

                      and

                      bull Parseval Inequality

                      V

                      xPxPx VV 222xPxPx VV

                      2

                      1

                      22

                      1

                      22 aavxxxP

                      n

                      ii

                      n

                      iiV

                      Review of Linear Algebra (Cont)

                      (Real) Unitary Matrices with

                      bull Orthonormal Basis Matrix

                      (So All of Above Applies)

                      ddU IUU t

                      Review of Linear Algebra (Cont)

                      (Real) Unitary Matrices with

                      bull Orthonormal Basis Matrix

                      (So All of Above Applies)

                      bull Note Transformrsquon is Distance Preserving

                      ddU IUU t

                      yxdyxyxyxUyUxUdn

                      i ii 2

                      1

                      Review of Linear Algebra (Cont)

                      (Real) Unitary Matrices with

                      bull Orthonormal Basis Matrix

                      (So All of Above Applies)

                      bull Note Transformrsquon is Distance Preserving

                      bull Lin Trans (Mult by ) is ~ Rotation

                      bull But also Includes ldquoMirror Imagesrdquo

                      ddU IUU t

                      yxdyxyxyxUyUxUdn

                      i ii 2

                      1

                      U

                      Review of Linear Algebra (Cont)

                      Singular Value Decomposition (SVD)

                      For a Matrix

                      Find

                      ndX

                      Review of Linear Algebra (Cont)

                      Singular Value Decomposition (SVD)

                      For a Matrix

                      Find a Diagonal Matrix

                      with Entries

                      called Singular Values

                      ndX

                      ndS

                      )min(1 ndss

                      Review of Linear Algebra (Cont)

                      Singular Value Decomposition (SVD)

                      For a Matrix

                      Find a Diagonal Matrix

                      with Entries

                      called Singular Values

                      And Unitary (Rotation) Matrices

                      (recall )

                      ndX

                      ndS

                      )min(1 ndss

                      ddU nnV

                      IVVUU tt

                      Review of Linear Algebra (Cont)

                      Singular Value Decomposition (SVD)

                      For a Matrix

                      Find a Diagonal Matrix

                      with Entries

                      called Singular Values

                      And Unitary (Rotation) Matrices

                      (recall )

                      So That

                      ndX

                      ndS

                      )min(1 ndss

                      ddU nnV

                      IVVUU tt tUSVX

                      Review of Linear Algebra (Cont)

                      Intuition behind Singular Value Decomposition

                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                      vVSUvVSUvX tt

                      Review of Linear Algebra (Cont)

                      Intuition behind Singular Value Decomposition

                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                      bull First rotate

                      X

                      vVSUvVSUvX tt

                      Review of Linear Algebra (Cont)

                      Intuition behind Singular Value Decomposition

                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                      bull First rotate

                      bull Second rescale coordinate axes (by )

                      X

                      vVSUvVSUvX tt

                      is

                      Review of Linear Algebra (Cont)

                      Intuition behind Singular Value Decomposition

                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                      bull First rotate

                      bull Second rescale coordinate axes (by )

                      bull Third rotate again

                      X

                      vVSUvVSUvX tt

                      is

                      Review of Linear Algebra (Cont)

                      Intuition behind Singular Value Decomposition

                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                      bull First rotate

                      bull Second rescale coordinate axes (by )

                      bull Third rotate again

                      bull ie have diagonalized the transformation

                      X

                      vVSUvVSUvX tt

                      is

                      SVD Compact Representation

                      Useful Labeling

                      Singular Values in Increasing Order

                      Review of Linear Algebra (Cont)

                      )min(1 dnss

                      SVD Compact Representation

                      Useful Labeling

                      Singular Values in Increasing Order

                      Note singular values = 0 can be omitted

                      (Since do ldquo0-Stretchingrdquo)

                      Review of Linear Algebra (Cont)

                      )min(1 dnss

                      Review of Linear Algebra (Cont)

                      r

                      SVD Compact Representation

                      Useful Labeling

                      Singular Values in Increasing Order

                      Note singular values = 0 can be omitted

                      Let = of positive singular values

                      )min(1 dnss

                      Review of Linear Algebra (Cont)

                      r

                      SVD Compact Representation

                      Useful Labeling

                      Singular Values in Increasing Order

                      Note singular values = 0 can be omitted

                      Let = of positive singular values

                      Then

                      Where are truncations of

                      trnrrrd VSUX

                      VSU

                      )min(1 dnss

                      Review of Linear Algebra (Cont)

                      SVD Full Representation

                      =

                      Graphics Display Assumes

                      ndX ddU ndS nn

                      tV

                      nd

                      Review of Linear Algebra (Cont)

                      SVD Full Representation

                      =

                      Full Rank Basis Matrix

                      ndX ddU ndS nn

                      tV

                      Review of Linear Algebra (Cont)

                      SVD Full Representation

                      =

                      Full Rank Basis Matrix

                      All 0s in Bottom

                      ndX ddU ndS nn

                      tV

                      Review of Linear Algebra (Cont)

                      SVD Reduced Representation

                      =

                      These Columns Get 0ed Out

                      ndX ddU nnS nn

                      tV

                      nnd 0

                      Review of Linear Algebra (Cont)

                      SVD Reduced Representation

                      =ndX ndU nnS nn

                      tV

                      Review of Linear Algebra (Cont)

                      SVD Reduced Representation

                      =

                      Also Some of These May be 0

                      ndX ndU nnS nn

                      tV

                      Review of Linear Algebra (Cont)

                      SVD Compact Representation

                      = ndX rdU

                      rrS nrtV

                      0

                      Review of Linear Algebra (Cont)

                      SVD Compact Representation

                      =

                      These Get 0ed Out

                      ndX rdU

                      rrS nrtV

                      0

                      Review of Linear Algebra (Cont)

                      SVD Compact Representation

                      = ndX rdU

                      rrS nrtV

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition

                      For a (Symmetric) Square Matrix

                      Find

                      ddX

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition

                      For a (Symmetric) Square Matrix

                      Find a Diagonal Matrix

                      Called Eigenvalues

                      Convenient Ordering

                      ddX

                      d

                      D

                      0

                      01

                      n 1

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition

                      For a (Symmetric) Square Matrix

                      Find a Diagonal Matrix

                      And an Orthonormal Matrix

                      (ie )

                      ddX

                      d

                      D

                      0

                      01

                      ddB

                      ddtt IBBBB

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition

                      For a (Symmetric) Square Matrix

                      Find a Diagonal Matrix

                      And an Orthonormal Matrix

                      (ie )

                      So that ie

                      ddX

                      d

                      D

                      0

                      01

                      ddB

                      ddtt IBBBB

                      DBBX tBDBX

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                      (looks similar)

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                      Complex (uses )

                      VU

                      1i

                      Review of Linear Algebra (Cont)

                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                      VU

                      X

                      BVU

                      1i

                      • Participant Presentations
                      • PCA to find clusters
                      • Statistical Smoothing
                      • Kernel Density Estimation
                      • Scatterplot Smoothing
                      • Statistical Smoothing (2)
                      • SiZer Background
                      • SiZer Background (2)
                      • SiZer Background (3)
                      • SiZer Background (4)
                      • SiZer Background (5)
                      • SiZer Overview
                      • PCA to find clusters (2)
                      • PCA to find clusters (3)
                      • PCA to find clusters (4)
                      • PCA to find clusters (5)
                      • PCA to find clusters (6)
                      • PCA to find clusters (7)
                      • Recall Yeast Cell Cycle Data
                      • Yeast Cell Cycle Data FDA View
                      • Yeast Cell Cycle Data FDA View (2)
                      • Yeast Cell Cycle Data FDA View (3)
                      • Yeast Cell Cycles Freq 2 Proj
                      • Frequency 2 Analysis
                      • Frequency 2 Analysis (2)
                      • Frequency 2 Analysis (3)
                      • Yeast Cell Cycle
                      • SiZer Study of Distrsquon of Angles
                      • Reclassification of Major Genes
                      • Compare to Previous Classifrsquon
                      • New Subpopulation View
                      • New Subpopulation View (2)
                      • Detailed Look at PCA
                      • PCA Rediscovery ndash Renaming
                      • PCA Rediscovery ndash Renaming (2)
                      • PCA Rediscovery ndash Renaming (3)
                      • PCA Rediscovery ndash Renaming (4)
                      • PCA Rediscovery ndash Renaming (5)
                      • An Interesting Historical Note
                      • An Interesting Historical Note (2)
                      • Detailed Look at PCA (2)
                      • Detailed Look at PCA (3)
                      • Review of Linear Algebra
                      • Review of Linear Algebra (2)
                      • Review of Linear Algebra (3)
                      • Review of Linear Algebra (Cont)
                      • Review of Linear Algebra (Cont) (2)
                      • Review of Linear Algebra (Cont) (3)
                      • Review of Linear Algebra (Cont) (4)
                      • Review of Linear Algebra (Cont) (5)
                      • Review of Linear Algebra (Cont) (6)
                      • Review of Linear Algebra (Cont) (7)
                      • Review of Linear Algebra (Cont) (8)
                      • Review of Linear Algebra (Cont) (9)
                      • Review of Linear Algebra (Cont) (10)
                      • Review of Linear Algebra (Cont) (11)
                      • Review of Linear Algebra (Cont) (12)
                      • Review of Linear Algebra (Cont) (13)
                      • Review of Linear Algebra (Cont) (14)
                      • Review of Linear Algebra (Cont) (15)
                      • Review of Linear Algebra (Cont) (16)
                      • Review of Linear Algebra (Cont) (17)
                      • Review of Linear Algebra (Cont) (18)
                      • Review of Linear Algebra (Cont) (19)
                      • Review of Linear Algebra (Cont) (20)
                      • Review of Linear Algebra (Cont) (21)
                      • Review of Linear Algebra (Cont) (22)
                      • Review of Linear Algebra (Cont) (23)
                      • Review of Linear Algebra (Cont) (24)
                      • Review of Linear Algebra (Cont) (25)
                      • Review of Linear Algebra (Cont) (26)
                      • Review of Linear Algebra (Cont) (27)
                      • Review of Linear Algebra (Cont) (28)
                      • Review of Linear Algebra (Cont) (29)
                      • Review of Linear Algebra (Cont) (30)
                      • Review of Linear Algebra (Cont) (31)
                      • Review of Linear Algebra (Cont) (32)
                      • Review of Linear Algebra (Cont) (33)
                      • Review of Linear Algebra (Cont) (34)
                      • Review of Linear Algebra (Cont) (35)
                      • Review of Linear Algebra (Cont) (36)
                      • Review of Linear Algebra (Cont) (37)
                      • Review of Linear Algebra (Cont) (38)
                      • Review of Linear Algebra (Cont) (39)
                      • Review of Linear Algebra (Cont) (40)
                      • Review of Linear Algebra (Cont) (41)
                      • Review of Linear Algebra (Cont) (42)
                      • Review of Linear Algebra (Cont) (43)
                      • Review of Linear Algebra (Cont) (44)
                      • Review of Linear Algebra (Cont) (45)
                      • Review of Linear Algebra (Cont) (46)
                      • Review of Linear Algebra (Cont) (47)
                      • Review of Linear Algebra (Cont) (48)
                      • Review of Linear Algebra (Cont) (49)
                      • Review of Linear Algebra (Cont) (50)
                      • Review of Linear Algebra (Cont) (51)
                      • Review of Linear Algebra (Cont) (52)
                      • Review of Linear Algebra (Cont) (53)
                      • Review of Linear Algebra (Cont) (54)
                      • Review of Linear Algebra (Cont) (55)
                      • Review of Linear Algebra (Cont) (56)
                      • Review of Linear Algebra (Cont) (57)
                      • Review of Linear Algebra (Cont) (58)
                      • Review of Linear Algebra (Cont) (59)
                      • Review of Linear Algebra (Cont) (60)
                      • Review of Linear Algebra (Cont) (61)
                      • Review of Linear Algebra (Cont) (62)
                      • Review of Linear Algebra (Cont) (63)
                      • Review of Linear Algebra (Cont) (64)
                      • Review of Linear Algebra (Cont) (65)
                      • Review of Linear Algebra (Cont) (66)
                      • Review of Linear Algebra (Cont) (67)
                      • Review of Linear Algebra (Cont) (68)
                      • Review of Linear Algebra (Cont) (69)
                      • Review of Linear Algebra (Cont) (70)
                      • Review of Linear Algebra (Cont) (71)
                      • Review of Linear Algebra (Cont) (72)
                      • Review of Linear Algebra (Cont) (73)
                      • Review of Linear Algebra (Cont) (74)
                      • Review of Linear Algebra (Cont) (75)
                      • Review of Linear Algebra (Cont) (76)
                      • Review of Linear Algebra (Cont) (77)
                      • Review of Linear Algebra (Cont) (78)
                      • Review of Linear Algebra (Cont) (79)
                      • Review of Linear Algebra (Cont) (80)
                      • Review of Linear Algebra (Cont) (81)

                        SiZer Overview

                        Would you like to try smoothing amp

                        SiZer

                        bull Marron Software Website as Before

                        bull In ldquoSmoothingrdquo Directory

                        ndash kdeSMm

                        ndash nprSMm

                        ndash sizerSMm

                        bull Recall ldquogtgt help sizerSMrdquo for

                        usage

                        PCA to find clustersReturn to PCA of Mass Flux Data

                        PCA to find clustersSiZer analysis of Mass Flux PC1

                        PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                        PCA to find clustersSiZer analysis of Mass Flux PC1

                        Also in Curvature

                        PCA to find clustersSiZer analysis of Mass Flux PC1

                        And in Other Comprsquos

                        PCA to find clustersSiZer analysis of Mass Flux PC1

                        Conclusion

                        bull Found 3 significant clusters

                        bull Correspond to 3 known ldquocloud typesrdquo

                        bull Worth deeper investigation

                        Recall Yeast Cell Cycle Data

                        bull ldquoGene Expressionrdquo ndash Micro-array data

                        bull Data (after major preprocessing) Expression ldquolevelrdquo of

                        bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                        10s)bull Interesting statistical issue

                        High Dimension Low Sample Size data

                        (HDLSS)

                        Yeast Cell Cycle Data FDA View

                        Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                        Yeast Cell Cycle Data FDA View

                        Periodic genes

                        Naiumlve

                        approach

                        Simple PCA

                        Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                        ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                        ldquovariationrdquobull PCA finds ldquodirections of maximal

                        variationrdquobull Often but not always same as

                        ldquointeresting directionsrdquobull Here need better approach to study

                        periodicities

                        Yeast Cell Cycles Freq 2 Proj

                        PCA on

                        Freq 2

                        Periodic

                        Component

                        Of Data

                        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                        Approach from Zhao Marron amp Wells (2004)

                        Frequency 2 Analysis

                        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                        classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                        notable differencesbull Now try to improve ldquophase classificationrdquo

                        Yeast Cell CycleRevisit ldquophase classificationrdquo

                        approachbull Use outer 200 genes

                        (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                        (finds significant bumps etc in histogram)

                        bull Carefully redrew boundariesbull Check by studying kde angles

                        SiZer Study of Distrsquon of Angles

                        Reclassification of Major Genes

                        Compare to Previous Classifrsquon

                        New Subpopulation View

                        New Subpopulation View

                        NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                        Detailed Look at PCA

                        Now Study ldquoFolklorerdquo More Carefully

                        bull BackGround

                        bull History

                        bull Underpinnings

                        (Mathematical amp Computational)

                        Good Overall Reference Jolliffe (2002)

                        PCA Rediscovery ndash Renaming

                        Statistics Principal Component Analysis (PCA)

                        PCA Rediscovery ndash Renaming

                        Statistics Principal Component Analysis (PCA)

                        Social Sciences Factor Analysis (PCA is a subset)

                        PCA Rediscovery ndash Renaming

                        Statistics Principal Component Analysis (PCA)

                        Social Sciences Factor Analysis (PCA is a subset)

                        Probability Electrical EngKarhunen ndash Loeve expansion

                        PCA Rediscovery ndash Renaming

                        Statistics Principal Component Analysis (PCA)

                        Social Sciences Factor Analysis (PCA is a subset)

                        Probability Electrical EngKarhunen ndash Loeve expansion

                        Applied MathematicsProper Orthogonal Decomposition (POD)

                        PCA Rediscovery ndash Renaming

                        Statistics Principal Component Analysis (PCA)

                        Social Sciences Factor Analysis (PCA is a subset)

                        Probability Electrical EngKarhunen ndash Loeve expansion

                        Applied MathematicsProper Orthogonal Decomposition (POD)

                        Geo-Sciences Empirical Orthogonal Functions (EOF)

                        An Interesting Historical Note

                        The 1st () application of PCA to Functional

                        Data Analysis

                        An Interesting Historical Note

                        The 1st () application of PCA to Functional

                        Data Analysis

                        Rao (1958)

                        1st Paper with ldquoCurves as Data Objectsrdquo

                        viewpoint

                        Detailed Look at PCA

                        Three Important (amp Interesting) Viewpoints

                        1 Mathematics

                        2 Numerics

                        3 Statistics

                        Goal Study Interrelationships

                        Detailed Look at PCA

                        Three Important (amp Interesting) Viewpoints

                        1 Mathematics

                        2 Numerics

                        3 Statistics

                        1st Review Linear Alg and Multivar Prob

                        Review of Linear Algebra

                        Vector Space

                        bull set of ldquovectorsrdquo

                        bull and ldquoscalarsrdquo (coefficients)

                        xa

                        Review of Linear Algebra

                        Vector Space

                        bull set of ldquovectorsrdquo

                        bull and ldquoscalarsrdquo (coefficients)

                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                        ( in space)

                        xa

                        i

                        ii xa

                        Review of Linear Algebra

                        Vector Space

                        bull set of ldquovectorsrdquo

                        bull and ldquoscalarsrdquo (coefficients)

                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                        ( in space)

                        eg

                        ldquo dim Euclidrsquon spacerdquo

                        xa

                        i

                        ii xa

                        d

                        d

                        d xx

                        x

                        x

                        x 1

                        1

                        d

                        Review of Linear Algebra (Cont)

                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                        Review of Linear Algebra (Cont)

                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                        Review of Linear Algebra (Cont)

                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                        Note Planes not Through the Origin

                        are not Subspaces

                        (Do not Contain )00 x

                        Review of Linear Algebra (Cont)

                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                        (all Linear Combos of them =

                        = Containing Hyperplane

                        through Origin)

                        Review of Linear Algebra (Cont)

                        Basis of Subspace Set of Vectors that

                        bull Span ie Everything is a Lin Com of them

                        bull are Linearly Indeprsquot ie Lin Com is Unique

                        Review of Linear Algebra (Cont)

                        Basis of Subspace Set of Vectors that

                        bull Span ie Everything is a Lin Com of them

                        bull are Linearly Indeprsquot ie Lin Com is Unique

                        bull eg ldquoUnit Vector Basisrdquo

                        bull Since

                        d

                        1

                        0

                        0

                        0

                        1

                        0

                        0

                        0

                        1

                        1

                        0

                        0

                        0

                        1

                        0

                        0

                        0

                        1

                        212

                        1

                        d

                        d

                        xxx

                        x

                        x

                        x

                        Review of Linear Algebra (Cont)

                        Basis Matrix of subspace of

                        Given a basis

                        create matrix of columns

                        dnvv 1

                        nddnd

                        n

                        n

                        vv

                        vv

                        vvB

                        1

                        111

                        1

                        Review of Linear Algebra (Cont)

                        Then linear combo is a matrix multiplicatrsquon

                        where

                        n

                        iii aBva

                        1

                        na

                        a

                        a 1

                        Review of Linear Algebra (Cont)

                        Then linear combo is a matrix multiplicatrsquon

                        where

                        Note Right Multiplication Gives

                        Linear Combination of Column Vectors

                        n

                        iii aBva

                        1

                        na

                        a

                        a 1

                        Review of Linear Algebra (Cont)

                        Then linear combo is a matrix multiplicatrsquon

                        where

                        Check sizes

                        n

                        iii aBva

                        1

                        na

                        a

                        a 1

                        )1()(1 nndd

                        Review of Linear Algebra (Cont)

                        Aside on Matrix Multiplication (linear transformatrsquon)

                        For matrices

                        Define the Matrix Product

                        mkk

                        m

                        aa

                        aa

                        A

                        1

                        111

                        nmm

                        n

                        bb

                        bb

                        B

                        1

                        111

                        m

                        iniik

                        m

                        iiik

                        m

                        inii

                        m

                        iii

                        baba

                        baba

                        AB

                        1

                        11

                        11

                        111

                        Review of Linear Algebra (Cont)

                        Aside on Matrix Multiplication (linear transformatrsquon)

                        For matrices

                        Define the Matrix Product

                        (Inner Products of Rows With Columns )

                        mkk

                        m

                        aa

                        aa

                        A

                        1

                        111

                        nmm

                        n

                        bb

                        bb

                        B

                        1

                        111

                        m

                        iniik

                        m

                        iiik

                        m

                        inii

                        m

                        iii

                        baba

                        baba

                        AB

                        1

                        11

                        11

                        111

                        A B

                        Review of Linear Algebra (Cont)

                        Aside on Matrix Multiplication (linear transformatrsquon)

                        For matrices

                        Define the Matrix Product

                        (Inner Products of Rows With Columns )

                        (Composition of Linear Transformations)

                        mkk

                        m

                        aa

                        aa

                        A

                        1

                        111

                        nmm

                        n

                        bb

                        bb

                        B

                        1

                        111

                        m

                        iniik

                        m

                        iiik

                        m

                        inii

                        m

                        iii

                        baba

                        baba

                        AB

                        1

                        11

                        11

                        111

                        A B

                        Review of Linear Algebra (Cont)

                        Aside on Matrix Multiplication (linear transformatrsquon)

                        For matrices

                        Define the Matrix Product

                        (Inner Products of Rows With Columns )

                        (Composition of Linear Transformations)

                        Often Useful to Check Sizes

                        mkk

                        m

                        aa

                        aa

                        A

                        1

                        111

                        nmm

                        n

                        bb

                        bb

                        B

                        1

                        111

                        m

                        iniik

                        m

                        iiik

                        m

                        inii

                        m

                        iii

                        baba

                        baba

                        AB

                        1

                        11

                        11

                        111

                        nmmknk

                        A B

                        Review of Linear Algebra (Cont)

                        Aside on Matrix Multiplication (linear transformatrsquon)

                        For matrices

                        Define the Matrix Product

                        (Inner Products of Rows With Columns )

                        (Composition of Linear Transformations)

                        Often Useful to Check Sizes

                        mkk

                        m

                        aa

                        aa

                        A

                        1

                        111

                        nmm

                        n

                        bb

                        bb

                        B

                        1

                        111

                        m

                        iniik

                        m

                        iiik

                        m

                        inii

                        m

                        iii

                        baba

                        baba

                        AB

                        1

                        11

                        11

                        111

                        nmmknk

                        A B

                        Review of Linear Algebra (Cont)

                        Matrix Trace

                        bull For a Square Matrix

                        bull Define

                        mmm

                        m

                        aa

                        aa

                        A

                        1

                        111

                        m

                        iiiaAtr

                        1)(

                        Review of Linear Algebra (Cont)

                        Matrix Trace

                        bull For a Square Matrix

                        bull Define

                        bull Trace Commutes with Matrix Multiplication

                        mmm

                        m

                        aa

                        aa

                        A

                        1

                        111

                        m

                        iiiaAtr

                        1)(

                        BAtrABtr

                        Review of Linear Algebra (Cont)

                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                        bull Number of Elements in a Basis (Unique)

                        Review of Linear Algebra (Cont)

                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                        bull Number of Elements in a Basis (Unique)

                        bull (Use Basis Above)

                        bull eg dim of a line is 1

                        bull eg dim of a plane is 2

                        dd dim

                        Review of Linear Algebra (Cont)

                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                        bull Number of Elements in a Basis (Unique)

                        bull (Use Basis Above)

                        bull eg dim of a line is 1

                        bull eg dim of a plane is 2

                        bull Dimension is ldquoDegrees of Freedomrdquo

                        (in Statistical Uses eg ANOVA)

                        dd dim

                        Review of Linear Algebra (Cont)

                        Norm of a Vector

                        bull in d 21

                        21

                        1

                        2 xxxx td

                        jj

                        Review of Linear Algebra (Cont)

                        Norm of a Vector

                        bull in

                        bull Idea length of the vector

                        d 2121

                        1

                        2 xxxx td

                        jj

                        Review of Linear Algebra (Cont)

                        Norm of a Vector

                        bull in

                        bull Idea length of the vector

                        bull Note strange properties for high

                        eg ldquolength of diagonal of unit cuberdquo =

                        d 2121

                        1

                        2 xxxx td

                        jj

                        d

                        d

                        Review of Linear Algebra (Cont)

                        Norm of a Vector (cont)

                        bull Length Normalized Vector

                        (has Length 1 thus on Surf of Unit Sphere

                        amp is a Direction Vector)

                        x

                        x

                        Review of Linear Algebra (Cont)

                        Norm of a Vector (cont)

                        bull Length Normalized Vector

                        (has Length 1 thus on Surf of Unit Sphere

                        amp is a Direction Vector)

                        bull Define Distance as

                        x

                        x

                        yxyxyxyxd t

                        Review of Linear Algebra (Cont)

                        Inner (Dot Scalar) Product

                        bull for Vectors and

                        yxyxyx td

                        jjj

                        1

                        x y

                        Review of Linear Algebra (Cont)

                        Inner (Dot Scalar) Product

                        bull for Vectors and

                        bull Related to Norm via

                        yxyxyx td

                        jjj

                        1

                        x y

                        21 xxx

                        Review of Linear Algebra (Cont)

                        Inner (Dot Scalar) Product (cont)

                        bull measures ldquoangle between and rdquo as

                        yyxx

                        yx

                        yx

                        yxyxangle

                        tt

                        t

                        11 cos

                        cos

                        x y

                        Review of Linear Algebra (Cont)

                        Inner (Dot Scalar) Product (cont)

                        bull measures ldquoangle between and rdquo as

                        bull key to Orthogonality ie Perpendiculrsquoty

                        if and only if

                        yyxx

                        yx

                        yx

                        yxyxangle

                        tt

                        t

                        11 cos

                        cos

                        x y

                        yx 0 yx

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis

                        bull All Orthogonal to each other

                        ie for

                        bull All have Length 1

                        ie for

                        nvv 1

                        1 ii vv

                        0 ii vv ii

                        ni 1

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis (cont)

                        bull Spectral Representation

                        where

                        nvv 1

                        n

                        iii vax

                        1

                        ii vxa

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis (cont)

                        bull Spectral Representation

                        where

                        (Coefficient is Inner Product Cool Notation)

                        nvv 1

                        n

                        iii vax

                        1

                        ii vxa

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis (cont)

                        bull Spectral Representation

                        where

                        Check

                        nvv 1

                        n

                        iii vax

                        1

                        ii vxa

                        iii

                        n

                        iii

                        n

                        iiii avvavvavx

                        1

                        1

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis (cont)

                        bull Spectral Representation

                        where

                        Check

                        bull Matrix Notation where ie

                        For the Basis Matrix

                        nvv 1

                        n

                        iii vax

                        1

                        ii vxa

                        iii

                        n

                        iii

                        n

                        iiii avvavvavx

                        1

                        1

                        aBx Bxa tt xBa t

                        nvvB 1

                        Review of Linear Algebra (Cont)

                        Orthonormal Basis (cont)

                        bull Spectral Representation

                        where

                        Check

                        bull Matrix Notation where ie

                        is called transform of

                        (eg Fourier or Wavelet)

                        nvv 1

                        n

                        iii vax

                        1

                        ii vxa

                        iii

                        n

                        iii

                        n

                        iiii avvavvavx

                        1

                        1

                        aBx Bxa tt xBa t

                        xa

                        Review of Linear Algebra (Cont)

                        Parseval identity for

                        in subsp genrsquod by o n basis

                        x

                        nvv 1

                        2

                        1

                        22

                        1

                        2 aavxx

                        n

                        ii

                        n

                        ii

                        Review of Linear Algebra (Cont)

                        Parseval identity for

                        in subsp genrsquod by o n basis

                        bull Pythagorean theorem

                        bull ldquoDecomposition of Energyrdquo

                        bull ANOVA - sums of squares

                        x

                        nvv 1

                        2

                        1

                        22

                        1

                        2 aavxx

                        n

                        ii

                        n

                        ii

                        Review of Linear Algebra (Cont)

                        Parseval identity for

                        in subsp genrsquod by o n basis

                        bull Pythagorean theorem

                        bull ldquoDecomposition of Energyrdquo

                        bull ANOVA - sums of squares

                        bull Transform has same length as

                        ie ldquorotation in rdquo

                        x

                        nvv 1

                        2

                        1

                        22

                        1

                        2 aavxx

                        n

                        ii

                        n

                        ii

                        a xd

                        Projection of a Vector onto a Subspace

                        bull Idea Member of that is Closest to

                        (ie ldquoBest Approxrsquonrdquo)

                        Review of Linear Algebra (Cont)x

                        xV

                        V

                        Projection of a Vector onto a Subspace

                        bull Idea Member of that is Closest to

                        (ie ldquoBest Approxrsquonrdquo)

                        bull Find that Solves

                        (ldquoLeast Squaresrdquo)

                        Review of Linear Algebra (Cont)x

                        xV

                        V

                        VxPV vxVv

                        min

                        Projection of a Vector onto a Subspace

                        bull Idea Member of that is Closest to

                        (ie ldquoBest Approxrsquonrdquo)

                        bull Find that Solves

                        (ldquoLeast Squaresrdquo)

                        bull For Inner Product (Hilbert) Space

                        Exists and is Unique

                        Review of Linear Algebra (Cont)x

                        xV

                        V

                        VxPV vxVv

                        min

                        xPV

                        Projection of a Vector onto a Subspace (cont)

                        bull General Solution in for Basis Matrix

                        Review of Linear Algebra (Cont)

                        d VB

                        xBBBBxP tVV

                        tVVV

                        1

                        Projection of a Vector onto a Subspace (cont)

                        bull General Solution in for Basis Matrix

                        bull So Projrsquon Operator is Matrix Multrsquon

                        (thus projection is another linear operation)

                        Review of Linear Algebra (Cont)

                        d VB

                        xBBBBxP tVV

                        tVVV

                        1

                        tVV

                        tVVV BBBBP

                        1

                        Projection of a Vector onto a Subspace (cont)

                        bull General Solution in for Basis Matrix

                        bull So Projrsquon Operator is Matrix Multrsquon

                        (thus projection is another linear operation)

                        (note same operation underlies least squares)

                        Review of Linear Algebra (Cont)

                        d VB

                        xBBBBxP tVV

                        tVVV

                        1

                        tVV

                        tVVV BBBBP

                        1

                        Review of Linear Algebra (Cont)

                        Projection using Orthonormal Basis

                        bull Basis Matrix is Orthonormal

                        nnVtV IBB

                        10

                        01

                        1

                        111

                        1

                        1

                        nnn

                        n

                        ntn

                        t

                        vvvv

                        vvvv

                        vv

                        v

                        v

                        nvv 1

                        Review of Linear Algebra (Cont)

                        Projection using Orthonormal Basis

                        bull Basis Matrix is Orthonormal

                        bull So =

                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                        (Recall Right Multrsquon)

                        nnVtV IBB

                        10

                        01

                        1

                        111

                        1

                        1

                        nnn

                        n

                        ntn

                        t

                        vvvv

                        vvvv

                        vv

                        v

                        v

                        xBBxP tVVV

                        x V

                        nvv 1

                        Review of Linear Algebra (Cont)

                        Projection using Orthonormal Basis (cont)

                        bull For Orthogonal Complement

                        and

                        V

                        xPxPx VV 222xPxPx VV

                        Review of Linear Algebra (Cont)

                        Projection using Orthonormal Basis (cont)

                        bull For Orthogonal Complement

                        and

                        bull Parseval Inequality

                        V

                        xPxPx VV 222xPxPx VV

                        2

                        1

                        22

                        1

                        22 aavxxxP

                        n

                        ii

                        n

                        iiV

                        Review of Linear Algebra (Cont)

                        (Real) Unitary Matrices with

                        bull Orthonormal Basis Matrix

                        (So All of Above Applies)

                        ddU IUU t

                        Review of Linear Algebra (Cont)

                        (Real) Unitary Matrices with

                        bull Orthonormal Basis Matrix

                        (So All of Above Applies)

                        bull Note Transformrsquon is Distance Preserving

                        ddU IUU t

                        yxdyxyxyxUyUxUdn

                        i ii 2

                        1

                        Review of Linear Algebra (Cont)

                        (Real) Unitary Matrices with

                        bull Orthonormal Basis Matrix

                        (So All of Above Applies)

                        bull Note Transformrsquon is Distance Preserving

                        bull Lin Trans (Mult by ) is ~ Rotation

                        bull But also Includes ldquoMirror Imagesrdquo

                        ddU IUU t

                        yxdyxyxyxUyUxUdn

                        i ii 2

                        1

                        U

                        Review of Linear Algebra (Cont)

                        Singular Value Decomposition (SVD)

                        For a Matrix

                        Find

                        ndX

                        Review of Linear Algebra (Cont)

                        Singular Value Decomposition (SVD)

                        For a Matrix

                        Find a Diagonal Matrix

                        with Entries

                        called Singular Values

                        ndX

                        ndS

                        )min(1 ndss

                        Review of Linear Algebra (Cont)

                        Singular Value Decomposition (SVD)

                        For a Matrix

                        Find a Diagonal Matrix

                        with Entries

                        called Singular Values

                        And Unitary (Rotation) Matrices

                        (recall )

                        ndX

                        ndS

                        )min(1 ndss

                        ddU nnV

                        IVVUU tt

                        Review of Linear Algebra (Cont)

                        Singular Value Decomposition (SVD)

                        For a Matrix

                        Find a Diagonal Matrix

                        with Entries

                        called Singular Values

                        And Unitary (Rotation) Matrices

                        (recall )

                        So That

                        ndX

                        ndS

                        )min(1 ndss

                        ddU nnV

                        IVVUU tt tUSVX

                        Review of Linear Algebra (Cont)

                        Intuition behind Singular Value Decomposition

                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                        vVSUvVSUvX tt

                        Review of Linear Algebra (Cont)

                        Intuition behind Singular Value Decomposition

                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                        bull First rotate

                        X

                        vVSUvVSUvX tt

                        Review of Linear Algebra (Cont)

                        Intuition behind Singular Value Decomposition

                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                        bull First rotate

                        bull Second rescale coordinate axes (by )

                        X

                        vVSUvVSUvX tt

                        is

                        Review of Linear Algebra (Cont)

                        Intuition behind Singular Value Decomposition

                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                        bull First rotate

                        bull Second rescale coordinate axes (by )

                        bull Third rotate again

                        X

                        vVSUvVSUvX tt

                        is

                        Review of Linear Algebra (Cont)

                        Intuition behind Singular Value Decomposition

                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                        bull First rotate

                        bull Second rescale coordinate axes (by )

                        bull Third rotate again

                        bull ie have diagonalized the transformation

                        X

                        vVSUvVSUvX tt

                        is

                        SVD Compact Representation

                        Useful Labeling

                        Singular Values in Increasing Order

                        Review of Linear Algebra (Cont)

                        )min(1 dnss

                        SVD Compact Representation

                        Useful Labeling

                        Singular Values in Increasing Order

                        Note singular values = 0 can be omitted

                        (Since do ldquo0-Stretchingrdquo)

                        Review of Linear Algebra (Cont)

                        )min(1 dnss

                        Review of Linear Algebra (Cont)

                        r

                        SVD Compact Representation

                        Useful Labeling

                        Singular Values in Increasing Order

                        Note singular values = 0 can be omitted

                        Let = of positive singular values

                        )min(1 dnss

                        Review of Linear Algebra (Cont)

                        r

                        SVD Compact Representation

                        Useful Labeling

                        Singular Values in Increasing Order

                        Note singular values = 0 can be omitted

                        Let = of positive singular values

                        Then

                        Where are truncations of

                        trnrrrd VSUX

                        VSU

                        )min(1 dnss

                        Review of Linear Algebra (Cont)

                        SVD Full Representation

                        =

                        Graphics Display Assumes

                        ndX ddU ndS nn

                        tV

                        nd

                        Review of Linear Algebra (Cont)

                        SVD Full Representation

                        =

                        Full Rank Basis Matrix

                        ndX ddU ndS nn

                        tV

                        Review of Linear Algebra (Cont)

                        SVD Full Representation

                        =

                        Full Rank Basis Matrix

                        All 0s in Bottom

                        ndX ddU ndS nn

                        tV

                        Review of Linear Algebra (Cont)

                        SVD Reduced Representation

                        =

                        These Columns Get 0ed Out

                        ndX ddU nnS nn

                        tV

                        nnd 0

                        Review of Linear Algebra (Cont)

                        SVD Reduced Representation

                        =ndX ndU nnS nn

                        tV

                        Review of Linear Algebra (Cont)

                        SVD Reduced Representation

                        =

                        Also Some of These May be 0

                        ndX ndU nnS nn

                        tV

                        Review of Linear Algebra (Cont)

                        SVD Compact Representation

                        = ndX rdU

                        rrS nrtV

                        0

                        Review of Linear Algebra (Cont)

                        SVD Compact Representation

                        =

                        These Get 0ed Out

                        ndX rdU

                        rrS nrtV

                        0

                        Review of Linear Algebra (Cont)

                        SVD Compact Representation

                        = ndX rdU

                        rrS nrtV

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition

                        For a (Symmetric) Square Matrix

                        Find

                        ddX

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition

                        For a (Symmetric) Square Matrix

                        Find a Diagonal Matrix

                        Called Eigenvalues

                        Convenient Ordering

                        ddX

                        d

                        D

                        0

                        01

                        n 1

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition

                        For a (Symmetric) Square Matrix

                        Find a Diagonal Matrix

                        And an Orthonormal Matrix

                        (ie )

                        ddX

                        d

                        D

                        0

                        01

                        ddB

                        ddtt IBBBB

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition

                        For a (Symmetric) Square Matrix

                        Find a Diagonal Matrix

                        And an Orthonormal Matrix

                        (ie )

                        So that ie

                        ddX

                        d

                        D

                        0

                        01

                        ddB

                        ddtt IBBBB

                        DBBX tBDBX

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                        (looks similar)

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                        Complex (uses )

                        VU

                        1i

                        Review of Linear Algebra (Cont)

                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                        VU

                        X

                        BVU

                        1i

                        • Participant Presentations
                        • PCA to find clusters
                        • Statistical Smoothing
                        • Kernel Density Estimation
                        • Scatterplot Smoothing
                        • Statistical Smoothing (2)
                        • SiZer Background
                        • SiZer Background (2)
                        • SiZer Background (3)
                        • SiZer Background (4)
                        • SiZer Background (5)
                        • SiZer Overview
                        • PCA to find clusters (2)
                        • PCA to find clusters (3)
                        • PCA to find clusters (4)
                        • PCA to find clusters (5)
                        • PCA to find clusters (6)
                        • PCA to find clusters (7)
                        • Recall Yeast Cell Cycle Data
                        • Yeast Cell Cycle Data FDA View
                        • Yeast Cell Cycle Data FDA View (2)
                        • Yeast Cell Cycle Data FDA View (3)
                        • Yeast Cell Cycles Freq 2 Proj
                        • Frequency 2 Analysis
                        • Frequency 2 Analysis (2)
                        • Frequency 2 Analysis (3)
                        • Yeast Cell Cycle
                        • SiZer Study of Distrsquon of Angles
                        • Reclassification of Major Genes
                        • Compare to Previous Classifrsquon
                        • New Subpopulation View
                        • New Subpopulation View (2)
                        • Detailed Look at PCA
                        • PCA Rediscovery ndash Renaming
                        • PCA Rediscovery ndash Renaming (2)
                        • PCA Rediscovery ndash Renaming (3)
                        • PCA Rediscovery ndash Renaming (4)
                        • PCA Rediscovery ndash Renaming (5)
                        • An Interesting Historical Note
                        • An Interesting Historical Note (2)
                        • Detailed Look at PCA (2)
                        • Detailed Look at PCA (3)
                        • Review of Linear Algebra
                        • Review of Linear Algebra (2)
                        • Review of Linear Algebra (3)
                        • Review of Linear Algebra (Cont)
                        • Review of Linear Algebra (Cont) (2)
                        • Review of Linear Algebra (Cont) (3)
                        • Review of Linear Algebra (Cont) (4)
                        • Review of Linear Algebra (Cont) (5)
                        • Review of Linear Algebra (Cont) (6)
                        • Review of Linear Algebra (Cont) (7)
                        • Review of Linear Algebra (Cont) (8)
                        • Review of Linear Algebra (Cont) (9)
                        • Review of Linear Algebra (Cont) (10)
                        • Review of Linear Algebra (Cont) (11)
                        • Review of Linear Algebra (Cont) (12)
                        • Review of Linear Algebra (Cont) (13)
                        • Review of Linear Algebra (Cont) (14)
                        • Review of Linear Algebra (Cont) (15)
                        • Review of Linear Algebra (Cont) (16)
                        • Review of Linear Algebra (Cont) (17)
                        • Review of Linear Algebra (Cont) (18)
                        • Review of Linear Algebra (Cont) (19)
                        • Review of Linear Algebra (Cont) (20)
                        • Review of Linear Algebra (Cont) (21)
                        • Review of Linear Algebra (Cont) (22)
                        • Review of Linear Algebra (Cont) (23)
                        • Review of Linear Algebra (Cont) (24)
                        • Review of Linear Algebra (Cont) (25)
                        • Review of Linear Algebra (Cont) (26)
                        • Review of Linear Algebra (Cont) (27)
                        • Review of Linear Algebra (Cont) (28)
                        • Review of Linear Algebra (Cont) (29)
                        • Review of Linear Algebra (Cont) (30)
                        • Review of Linear Algebra (Cont) (31)
                        • Review of Linear Algebra (Cont) (32)
                        • Review of Linear Algebra (Cont) (33)
                        • Review of Linear Algebra (Cont) (34)
                        • Review of Linear Algebra (Cont) (35)
                        • Review of Linear Algebra (Cont) (36)
                        • Review of Linear Algebra (Cont) (37)
                        • Review of Linear Algebra (Cont) (38)
                        • Review of Linear Algebra (Cont) (39)
                        • Review of Linear Algebra (Cont) (40)
                        • Review of Linear Algebra (Cont) (41)
                        • Review of Linear Algebra (Cont) (42)
                        • Review of Linear Algebra (Cont) (43)
                        • Review of Linear Algebra (Cont) (44)
                        • Review of Linear Algebra (Cont) (45)
                        • Review of Linear Algebra (Cont) (46)
                        • Review of Linear Algebra (Cont) (47)
                        • Review of Linear Algebra (Cont) (48)
                        • Review of Linear Algebra (Cont) (49)
                        • Review of Linear Algebra (Cont) (50)
                        • Review of Linear Algebra (Cont) (51)
                        • Review of Linear Algebra (Cont) (52)
                        • Review of Linear Algebra (Cont) (53)
                        • Review of Linear Algebra (Cont) (54)
                        • Review of Linear Algebra (Cont) (55)
                        • Review of Linear Algebra (Cont) (56)
                        • Review of Linear Algebra (Cont) (57)
                        • Review of Linear Algebra (Cont) (58)
                        • Review of Linear Algebra (Cont) (59)
                        • Review of Linear Algebra (Cont) (60)
                        • Review of Linear Algebra (Cont) (61)
                        • Review of Linear Algebra (Cont) (62)
                        • Review of Linear Algebra (Cont) (63)
                        • Review of Linear Algebra (Cont) (64)
                        • Review of Linear Algebra (Cont) (65)
                        • Review of Linear Algebra (Cont) (66)
                        • Review of Linear Algebra (Cont) (67)
                        • Review of Linear Algebra (Cont) (68)
                        • Review of Linear Algebra (Cont) (69)
                        • Review of Linear Algebra (Cont) (70)
                        • Review of Linear Algebra (Cont) (71)
                        • Review of Linear Algebra (Cont) (72)
                        • Review of Linear Algebra (Cont) (73)
                        • Review of Linear Algebra (Cont) (74)
                        • Review of Linear Algebra (Cont) (75)
                        • Review of Linear Algebra (Cont) (76)
                        • Review of Linear Algebra (Cont) (77)
                        • Review of Linear Algebra (Cont) (78)
                        • Review of Linear Algebra (Cont) (79)
                        • Review of Linear Algebra (Cont) (80)
                        • Review of Linear Algebra (Cont) (81)

                          PCA to find clustersReturn to PCA of Mass Flux Data

                          PCA to find clustersSiZer analysis of Mass Flux PC1

                          PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                          PCA to find clustersSiZer analysis of Mass Flux PC1

                          Also in Curvature

                          PCA to find clustersSiZer analysis of Mass Flux PC1

                          And in Other Comprsquos

                          PCA to find clustersSiZer analysis of Mass Flux PC1

                          Conclusion

                          bull Found 3 significant clusters

                          bull Correspond to 3 known ldquocloud typesrdquo

                          bull Worth deeper investigation

                          Recall Yeast Cell Cycle Data

                          bull ldquoGene Expressionrdquo ndash Micro-array data

                          bull Data (after major preprocessing) Expression ldquolevelrdquo of

                          bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                          10s)bull Interesting statistical issue

                          High Dimension Low Sample Size data

                          (HDLSS)

                          Yeast Cell Cycle Data FDA View

                          Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                          Yeast Cell Cycle Data FDA View

                          Periodic genes

                          Naiumlve

                          approach

                          Simple PCA

                          Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                          ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                          ldquovariationrdquobull PCA finds ldquodirections of maximal

                          variationrdquobull Often but not always same as

                          ldquointeresting directionsrdquobull Here need better approach to study

                          periodicities

                          Yeast Cell Cycles Freq 2 Proj

                          PCA on

                          Freq 2

                          Periodic

                          Component

                          Of Data

                          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                          Approach from Zhao Marron amp Wells (2004)

                          Frequency 2 Analysis

                          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                          classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                          notable differencesbull Now try to improve ldquophase classificationrdquo

                          Yeast Cell CycleRevisit ldquophase classificationrdquo

                          approachbull Use outer 200 genes

                          (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                          (finds significant bumps etc in histogram)

                          bull Carefully redrew boundariesbull Check by studying kde angles

                          SiZer Study of Distrsquon of Angles

                          Reclassification of Major Genes

                          Compare to Previous Classifrsquon

                          New Subpopulation View

                          New Subpopulation View

                          NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                          Detailed Look at PCA

                          Now Study ldquoFolklorerdquo More Carefully

                          bull BackGround

                          bull History

                          bull Underpinnings

                          (Mathematical amp Computational)

                          Good Overall Reference Jolliffe (2002)

                          PCA Rediscovery ndash Renaming

                          Statistics Principal Component Analysis (PCA)

                          PCA Rediscovery ndash Renaming

                          Statistics Principal Component Analysis (PCA)

                          Social Sciences Factor Analysis (PCA is a subset)

                          PCA Rediscovery ndash Renaming

                          Statistics Principal Component Analysis (PCA)

                          Social Sciences Factor Analysis (PCA is a subset)

                          Probability Electrical EngKarhunen ndash Loeve expansion

                          PCA Rediscovery ndash Renaming

                          Statistics Principal Component Analysis (PCA)

                          Social Sciences Factor Analysis (PCA is a subset)

                          Probability Electrical EngKarhunen ndash Loeve expansion

                          Applied MathematicsProper Orthogonal Decomposition (POD)

                          PCA Rediscovery ndash Renaming

                          Statistics Principal Component Analysis (PCA)

                          Social Sciences Factor Analysis (PCA is a subset)

                          Probability Electrical EngKarhunen ndash Loeve expansion

                          Applied MathematicsProper Orthogonal Decomposition (POD)

                          Geo-Sciences Empirical Orthogonal Functions (EOF)

                          An Interesting Historical Note

                          The 1st () application of PCA to Functional

                          Data Analysis

                          An Interesting Historical Note

                          The 1st () application of PCA to Functional

                          Data Analysis

                          Rao (1958)

                          1st Paper with ldquoCurves as Data Objectsrdquo

                          viewpoint

                          Detailed Look at PCA

                          Three Important (amp Interesting) Viewpoints

                          1 Mathematics

                          2 Numerics

                          3 Statistics

                          Goal Study Interrelationships

                          Detailed Look at PCA

                          Three Important (amp Interesting) Viewpoints

                          1 Mathematics

                          2 Numerics

                          3 Statistics

                          1st Review Linear Alg and Multivar Prob

                          Review of Linear Algebra

                          Vector Space

                          bull set of ldquovectorsrdquo

                          bull and ldquoscalarsrdquo (coefficients)

                          xa

                          Review of Linear Algebra

                          Vector Space

                          bull set of ldquovectorsrdquo

                          bull and ldquoscalarsrdquo (coefficients)

                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                          ( in space)

                          xa

                          i

                          ii xa

                          Review of Linear Algebra

                          Vector Space

                          bull set of ldquovectorsrdquo

                          bull and ldquoscalarsrdquo (coefficients)

                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                          ( in space)

                          eg

                          ldquo dim Euclidrsquon spacerdquo

                          xa

                          i

                          ii xa

                          d

                          d

                          d xx

                          x

                          x

                          x 1

                          1

                          d

                          Review of Linear Algebra (Cont)

                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                          Review of Linear Algebra (Cont)

                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                          Review of Linear Algebra (Cont)

                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                          Note Planes not Through the Origin

                          are not Subspaces

                          (Do not Contain )00 x

                          Review of Linear Algebra (Cont)

                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                          (all Linear Combos of them =

                          = Containing Hyperplane

                          through Origin)

                          Review of Linear Algebra (Cont)

                          Basis of Subspace Set of Vectors that

                          bull Span ie Everything is a Lin Com of them

                          bull are Linearly Indeprsquot ie Lin Com is Unique

                          Review of Linear Algebra (Cont)

                          Basis of Subspace Set of Vectors that

                          bull Span ie Everything is a Lin Com of them

                          bull are Linearly Indeprsquot ie Lin Com is Unique

                          bull eg ldquoUnit Vector Basisrdquo

                          bull Since

                          d

                          1

                          0

                          0

                          0

                          1

                          0

                          0

                          0

                          1

                          1

                          0

                          0

                          0

                          1

                          0

                          0

                          0

                          1

                          212

                          1

                          d

                          d

                          xxx

                          x

                          x

                          x

                          Review of Linear Algebra (Cont)

                          Basis Matrix of subspace of

                          Given a basis

                          create matrix of columns

                          dnvv 1

                          nddnd

                          n

                          n

                          vv

                          vv

                          vvB

                          1

                          111

                          1

                          Review of Linear Algebra (Cont)

                          Then linear combo is a matrix multiplicatrsquon

                          where

                          n

                          iii aBva

                          1

                          na

                          a

                          a 1

                          Review of Linear Algebra (Cont)

                          Then linear combo is a matrix multiplicatrsquon

                          where

                          Note Right Multiplication Gives

                          Linear Combination of Column Vectors

                          n

                          iii aBva

                          1

                          na

                          a

                          a 1

                          Review of Linear Algebra (Cont)

                          Then linear combo is a matrix multiplicatrsquon

                          where

                          Check sizes

                          n

                          iii aBva

                          1

                          na

                          a

                          a 1

                          )1()(1 nndd

                          Review of Linear Algebra (Cont)

                          Aside on Matrix Multiplication (linear transformatrsquon)

                          For matrices

                          Define the Matrix Product

                          mkk

                          m

                          aa

                          aa

                          A

                          1

                          111

                          nmm

                          n

                          bb

                          bb

                          B

                          1

                          111

                          m

                          iniik

                          m

                          iiik

                          m

                          inii

                          m

                          iii

                          baba

                          baba

                          AB

                          1

                          11

                          11

                          111

                          Review of Linear Algebra (Cont)

                          Aside on Matrix Multiplication (linear transformatrsquon)

                          For matrices

                          Define the Matrix Product

                          (Inner Products of Rows With Columns )

                          mkk

                          m

                          aa

                          aa

                          A

                          1

                          111

                          nmm

                          n

                          bb

                          bb

                          B

                          1

                          111

                          m

                          iniik

                          m

                          iiik

                          m

                          inii

                          m

                          iii

                          baba

                          baba

                          AB

                          1

                          11

                          11

                          111

                          A B

                          Review of Linear Algebra (Cont)

                          Aside on Matrix Multiplication (linear transformatrsquon)

                          For matrices

                          Define the Matrix Product

                          (Inner Products of Rows With Columns )

                          (Composition of Linear Transformations)

                          mkk

                          m

                          aa

                          aa

                          A

                          1

                          111

                          nmm

                          n

                          bb

                          bb

                          B

                          1

                          111

                          m

                          iniik

                          m

                          iiik

                          m

                          inii

                          m

                          iii

                          baba

                          baba

                          AB

                          1

                          11

                          11

                          111

                          A B

                          Review of Linear Algebra (Cont)

                          Aside on Matrix Multiplication (linear transformatrsquon)

                          For matrices

                          Define the Matrix Product

                          (Inner Products of Rows With Columns )

                          (Composition of Linear Transformations)

                          Often Useful to Check Sizes

                          mkk

                          m

                          aa

                          aa

                          A

                          1

                          111

                          nmm

                          n

                          bb

                          bb

                          B

                          1

                          111

                          m

                          iniik

                          m

                          iiik

                          m

                          inii

                          m

                          iii

                          baba

                          baba

                          AB

                          1

                          11

                          11

                          111

                          nmmknk

                          A B

                          Review of Linear Algebra (Cont)

                          Aside on Matrix Multiplication (linear transformatrsquon)

                          For matrices

                          Define the Matrix Product

                          (Inner Products of Rows With Columns )

                          (Composition of Linear Transformations)

                          Often Useful to Check Sizes

                          mkk

                          m

                          aa

                          aa

                          A

                          1

                          111

                          nmm

                          n

                          bb

                          bb

                          B

                          1

                          111

                          m

                          iniik

                          m

                          iiik

                          m

                          inii

                          m

                          iii

                          baba

                          baba

                          AB

                          1

                          11

                          11

                          111

                          nmmknk

                          A B

                          Review of Linear Algebra (Cont)

                          Matrix Trace

                          bull For a Square Matrix

                          bull Define

                          mmm

                          m

                          aa

                          aa

                          A

                          1

                          111

                          m

                          iiiaAtr

                          1)(

                          Review of Linear Algebra (Cont)

                          Matrix Trace

                          bull For a Square Matrix

                          bull Define

                          bull Trace Commutes with Matrix Multiplication

                          mmm

                          m

                          aa

                          aa

                          A

                          1

                          111

                          m

                          iiiaAtr

                          1)(

                          BAtrABtr

                          Review of Linear Algebra (Cont)

                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                          bull Number of Elements in a Basis (Unique)

                          Review of Linear Algebra (Cont)

                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                          bull Number of Elements in a Basis (Unique)

                          bull (Use Basis Above)

                          bull eg dim of a line is 1

                          bull eg dim of a plane is 2

                          dd dim

                          Review of Linear Algebra (Cont)

                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                          bull Number of Elements in a Basis (Unique)

                          bull (Use Basis Above)

                          bull eg dim of a line is 1

                          bull eg dim of a plane is 2

                          bull Dimension is ldquoDegrees of Freedomrdquo

                          (in Statistical Uses eg ANOVA)

                          dd dim

                          Review of Linear Algebra (Cont)

                          Norm of a Vector

                          bull in d 21

                          21

                          1

                          2 xxxx td

                          jj

                          Review of Linear Algebra (Cont)

                          Norm of a Vector

                          bull in

                          bull Idea length of the vector

                          d 2121

                          1

                          2 xxxx td

                          jj

                          Review of Linear Algebra (Cont)

                          Norm of a Vector

                          bull in

                          bull Idea length of the vector

                          bull Note strange properties for high

                          eg ldquolength of diagonal of unit cuberdquo =

                          d 2121

                          1

                          2 xxxx td

                          jj

                          d

                          d

                          Review of Linear Algebra (Cont)

                          Norm of a Vector (cont)

                          bull Length Normalized Vector

                          (has Length 1 thus on Surf of Unit Sphere

                          amp is a Direction Vector)

                          x

                          x

                          Review of Linear Algebra (Cont)

                          Norm of a Vector (cont)

                          bull Length Normalized Vector

                          (has Length 1 thus on Surf of Unit Sphere

                          amp is a Direction Vector)

                          bull Define Distance as

                          x

                          x

                          yxyxyxyxd t

                          Review of Linear Algebra (Cont)

                          Inner (Dot Scalar) Product

                          bull for Vectors and

                          yxyxyx td

                          jjj

                          1

                          x y

                          Review of Linear Algebra (Cont)

                          Inner (Dot Scalar) Product

                          bull for Vectors and

                          bull Related to Norm via

                          yxyxyx td

                          jjj

                          1

                          x y

                          21 xxx

                          Review of Linear Algebra (Cont)

                          Inner (Dot Scalar) Product (cont)

                          bull measures ldquoangle between and rdquo as

                          yyxx

                          yx

                          yx

                          yxyxangle

                          tt

                          t

                          11 cos

                          cos

                          x y

                          Review of Linear Algebra (Cont)

                          Inner (Dot Scalar) Product (cont)

                          bull measures ldquoangle between and rdquo as

                          bull key to Orthogonality ie Perpendiculrsquoty

                          if and only if

                          yyxx

                          yx

                          yx

                          yxyxangle

                          tt

                          t

                          11 cos

                          cos

                          x y

                          yx 0 yx

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis

                          bull All Orthogonal to each other

                          ie for

                          bull All have Length 1

                          ie for

                          nvv 1

                          1 ii vv

                          0 ii vv ii

                          ni 1

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis (cont)

                          bull Spectral Representation

                          where

                          nvv 1

                          n

                          iii vax

                          1

                          ii vxa

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis (cont)

                          bull Spectral Representation

                          where

                          (Coefficient is Inner Product Cool Notation)

                          nvv 1

                          n

                          iii vax

                          1

                          ii vxa

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis (cont)

                          bull Spectral Representation

                          where

                          Check

                          nvv 1

                          n

                          iii vax

                          1

                          ii vxa

                          iii

                          n

                          iii

                          n

                          iiii avvavvavx

                          1

                          1

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis (cont)

                          bull Spectral Representation

                          where

                          Check

                          bull Matrix Notation where ie

                          For the Basis Matrix

                          nvv 1

                          n

                          iii vax

                          1

                          ii vxa

                          iii

                          n

                          iii

                          n

                          iiii avvavvavx

                          1

                          1

                          aBx Bxa tt xBa t

                          nvvB 1

                          Review of Linear Algebra (Cont)

                          Orthonormal Basis (cont)

                          bull Spectral Representation

                          where

                          Check

                          bull Matrix Notation where ie

                          is called transform of

                          (eg Fourier or Wavelet)

                          nvv 1

                          n

                          iii vax

                          1

                          ii vxa

                          iii

                          n

                          iii

                          n

                          iiii avvavvavx

                          1

                          1

                          aBx Bxa tt xBa t

                          xa

                          Review of Linear Algebra (Cont)

                          Parseval identity for

                          in subsp genrsquod by o n basis

                          x

                          nvv 1

                          2

                          1

                          22

                          1

                          2 aavxx

                          n

                          ii

                          n

                          ii

                          Review of Linear Algebra (Cont)

                          Parseval identity for

                          in subsp genrsquod by o n basis

                          bull Pythagorean theorem

                          bull ldquoDecomposition of Energyrdquo

                          bull ANOVA - sums of squares

                          x

                          nvv 1

                          2

                          1

                          22

                          1

                          2 aavxx

                          n

                          ii

                          n

                          ii

                          Review of Linear Algebra (Cont)

                          Parseval identity for

                          in subsp genrsquod by o n basis

                          bull Pythagorean theorem

                          bull ldquoDecomposition of Energyrdquo

                          bull ANOVA - sums of squares

                          bull Transform has same length as

                          ie ldquorotation in rdquo

                          x

                          nvv 1

                          2

                          1

                          22

                          1

                          2 aavxx

                          n

                          ii

                          n

                          ii

                          a xd

                          Projection of a Vector onto a Subspace

                          bull Idea Member of that is Closest to

                          (ie ldquoBest Approxrsquonrdquo)

                          Review of Linear Algebra (Cont)x

                          xV

                          V

                          Projection of a Vector onto a Subspace

                          bull Idea Member of that is Closest to

                          (ie ldquoBest Approxrsquonrdquo)

                          bull Find that Solves

                          (ldquoLeast Squaresrdquo)

                          Review of Linear Algebra (Cont)x

                          xV

                          V

                          VxPV vxVv

                          min

                          Projection of a Vector onto a Subspace

                          bull Idea Member of that is Closest to

                          (ie ldquoBest Approxrsquonrdquo)

                          bull Find that Solves

                          (ldquoLeast Squaresrdquo)

                          bull For Inner Product (Hilbert) Space

                          Exists and is Unique

                          Review of Linear Algebra (Cont)x

                          xV

                          V

                          VxPV vxVv

                          min

                          xPV

                          Projection of a Vector onto a Subspace (cont)

                          bull General Solution in for Basis Matrix

                          Review of Linear Algebra (Cont)

                          d VB

                          xBBBBxP tVV

                          tVVV

                          1

                          Projection of a Vector onto a Subspace (cont)

                          bull General Solution in for Basis Matrix

                          bull So Projrsquon Operator is Matrix Multrsquon

                          (thus projection is another linear operation)

                          Review of Linear Algebra (Cont)

                          d VB

                          xBBBBxP tVV

                          tVVV

                          1

                          tVV

                          tVVV BBBBP

                          1

                          Projection of a Vector onto a Subspace (cont)

                          bull General Solution in for Basis Matrix

                          bull So Projrsquon Operator is Matrix Multrsquon

                          (thus projection is another linear operation)

                          (note same operation underlies least squares)

                          Review of Linear Algebra (Cont)

                          d VB

                          xBBBBxP tVV

                          tVVV

                          1

                          tVV

                          tVVV BBBBP

                          1

                          Review of Linear Algebra (Cont)

                          Projection using Orthonormal Basis

                          bull Basis Matrix is Orthonormal

                          nnVtV IBB

                          10

                          01

                          1

                          111

                          1

                          1

                          nnn

                          n

                          ntn

                          t

                          vvvv

                          vvvv

                          vv

                          v

                          v

                          nvv 1

                          Review of Linear Algebra (Cont)

                          Projection using Orthonormal Basis

                          bull Basis Matrix is Orthonormal

                          bull So =

                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                          (Recall Right Multrsquon)

                          nnVtV IBB

                          10

                          01

                          1

                          111

                          1

                          1

                          nnn

                          n

                          ntn

                          t

                          vvvv

                          vvvv

                          vv

                          v

                          v

                          xBBxP tVVV

                          x V

                          nvv 1

                          Review of Linear Algebra (Cont)

                          Projection using Orthonormal Basis (cont)

                          bull For Orthogonal Complement

                          and

                          V

                          xPxPx VV 222xPxPx VV

                          Review of Linear Algebra (Cont)

                          Projection using Orthonormal Basis (cont)

                          bull For Orthogonal Complement

                          and

                          bull Parseval Inequality

                          V

                          xPxPx VV 222xPxPx VV

                          2

                          1

                          22

                          1

                          22 aavxxxP

                          n

                          ii

                          n

                          iiV

                          Review of Linear Algebra (Cont)

                          (Real) Unitary Matrices with

                          bull Orthonormal Basis Matrix

                          (So All of Above Applies)

                          ddU IUU t

                          Review of Linear Algebra (Cont)

                          (Real) Unitary Matrices with

                          bull Orthonormal Basis Matrix

                          (So All of Above Applies)

                          bull Note Transformrsquon is Distance Preserving

                          ddU IUU t

                          yxdyxyxyxUyUxUdn

                          i ii 2

                          1

                          Review of Linear Algebra (Cont)

                          (Real) Unitary Matrices with

                          bull Orthonormal Basis Matrix

                          (So All of Above Applies)

                          bull Note Transformrsquon is Distance Preserving

                          bull Lin Trans (Mult by ) is ~ Rotation

                          bull But also Includes ldquoMirror Imagesrdquo

                          ddU IUU t

                          yxdyxyxyxUyUxUdn

                          i ii 2

                          1

                          U

                          Review of Linear Algebra (Cont)

                          Singular Value Decomposition (SVD)

                          For a Matrix

                          Find

                          ndX

                          Review of Linear Algebra (Cont)

                          Singular Value Decomposition (SVD)

                          For a Matrix

                          Find a Diagonal Matrix

                          with Entries

                          called Singular Values

                          ndX

                          ndS

                          )min(1 ndss

                          Review of Linear Algebra (Cont)

                          Singular Value Decomposition (SVD)

                          For a Matrix

                          Find a Diagonal Matrix

                          with Entries

                          called Singular Values

                          And Unitary (Rotation) Matrices

                          (recall )

                          ndX

                          ndS

                          )min(1 ndss

                          ddU nnV

                          IVVUU tt

                          Review of Linear Algebra (Cont)

                          Singular Value Decomposition (SVD)

                          For a Matrix

                          Find a Diagonal Matrix

                          with Entries

                          called Singular Values

                          And Unitary (Rotation) Matrices

                          (recall )

                          So That

                          ndX

                          ndS

                          )min(1 ndss

                          ddU nnV

                          IVVUU tt tUSVX

                          Review of Linear Algebra (Cont)

                          Intuition behind Singular Value Decomposition

                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                          vVSUvVSUvX tt

                          Review of Linear Algebra (Cont)

                          Intuition behind Singular Value Decomposition

                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                          bull First rotate

                          X

                          vVSUvVSUvX tt

                          Review of Linear Algebra (Cont)

                          Intuition behind Singular Value Decomposition

                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                          bull First rotate

                          bull Second rescale coordinate axes (by )

                          X

                          vVSUvVSUvX tt

                          is

                          Review of Linear Algebra (Cont)

                          Intuition behind Singular Value Decomposition

                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                          bull First rotate

                          bull Second rescale coordinate axes (by )

                          bull Third rotate again

                          X

                          vVSUvVSUvX tt

                          is

                          Review of Linear Algebra (Cont)

                          Intuition behind Singular Value Decomposition

                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                          bull First rotate

                          bull Second rescale coordinate axes (by )

                          bull Third rotate again

                          bull ie have diagonalized the transformation

                          X

                          vVSUvVSUvX tt

                          is

                          SVD Compact Representation

                          Useful Labeling

                          Singular Values in Increasing Order

                          Review of Linear Algebra (Cont)

                          )min(1 dnss

                          SVD Compact Representation

                          Useful Labeling

                          Singular Values in Increasing Order

                          Note singular values = 0 can be omitted

                          (Since do ldquo0-Stretchingrdquo)

                          Review of Linear Algebra (Cont)

                          )min(1 dnss

                          Review of Linear Algebra (Cont)

                          r

                          SVD Compact Representation

                          Useful Labeling

                          Singular Values in Increasing Order

                          Note singular values = 0 can be omitted

                          Let = of positive singular values

                          )min(1 dnss

                          Review of Linear Algebra (Cont)

                          r

                          SVD Compact Representation

                          Useful Labeling

                          Singular Values in Increasing Order

                          Note singular values = 0 can be omitted

                          Let = of positive singular values

                          Then

                          Where are truncations of

                          trnrrrd VSUX

                          VSU

                          )min(1 dnss

                          Review of Linear Algebra (Cont)

                          SVD Full Representation

                          =

                          Graphics Display Assumes

                          ndX ddU ndS nn

                          tV

                          nd

                          Review of Linear Algebra (Cont)

                          SVD Full Representation

                          =

                          Full Rank Basis Matrix

                          ndX ddU ndS nn

                          tV

                          Review of Linear Algebra (Cont)

                          SVD Full Representation

                          =

                          Full Rank Basis Matrix

                          All 0s in Bottom

                          ndX ddU ndS nn

                          tV

                          Review of Linear Algebra (Cont)

                          SVD Reduced Representation

                          =

                          These Columns Get 0ed Out

                          ndX ddU nnS nn

                          tV

                          nnd 0

                          Review of Linear Algebra (Cont)

                          SVD Reduced Representation

                          =ndX ndU nnS nn

                          tV

                          Review of Linear Algebra (Cont)

                          SVD Reduced Representation

                          =

                          Also Some of These May be 0

                          ndX ndU nnS nn

                          tV

                          Review of Linear Algebra (Cont)

                          SVD Compact Representation

                          = ndX rdU

                          rrS nrtV

                          0

                          Review of Linear Algebra (Cont)

                          SVD Compact Representation

                          =

                          These Get 0ed Out

                          ndX rdU

                          rrS nrtV

                          0

                          Review of Linear Algebra (Cont)

                          SVD Compact Representation

                          = ndX rdU

                          rrS nrtV

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition

                          For a (Symmetric) Square Matrix

                          Find

                          ddX

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition

                          For a (Symmetric) Square Matrix

                          Find a Diagonal Matrix

                          Called Eigenvalues

                          Convenient Ordering

                          ddX

                          d

                          D

                          0

                          01

                          n 1

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition

                          For a (Symmetric) Square Matrix

                          Find a Diagonal Matrix

                          And an Orthonormal Matrix

                          (ie )

                          ddX

                          d

                          D

                          0

                          01

                          ddB

                          ddtt IBBBB

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition

                          For a (Symmetric) Square Matrix

                          Find a Diagonal Matrix

                          And an Orthonormal Matrix

                          (ie )

                          So that ie

                          ddX

                          d

                          D

                          0

                          01

                          ddB

                          ddtt IBBBB

                          DBBX tBDBX

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                          (looks similar)

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                          Complex (uses )

                          VU

                          1i

                          Review of Linear Algebra (Cont)

                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                          VU

                          X

                          BVU

                          1i

                          • Participant Presentations
                          • PCA to find clusters
                          • Statistical Smoothing
                          • Kernel Density Estimation
                          • Scatterplot Smoothing
                          • Statistical Smoothing (2)
                          • SiZer Background
                          • SiZer Background (2)
                          • SiZer Background (3)
                          • SiZer Background (4)
                          • SiZer Background (5)
                          • SiZer Overview
                          • PCA to find clusters (2)
                          • PCA to find clusters (3)
                          • PCA to find clusters (4)
                          • PCA to find clusters (5)
                          • PCA to find clusters (6)
                          • PCA to find clusters (7)
                          • Recall Yeast Cell Cycle Data
                          • Yeast Cell Cycle Data FDA View
                          • Yeast Cell Cycle Data FDA View (2)
                          • Yeast Cell Cycle Data FDA View (3)
                          • Yeast Cell Cycles Freq 2 Proj
                          • Frequency 2 Analysis
                          • Frequency 2 Analysis (2)
                          • Frequency 2 Analysis (3)
                          • Yeast Cell Cycle
                          • SiZer Study of Distrsquon of Angles
                          • Reclassification of Major Genes
                          • Compare to Previous Classifrsquon
                          • New Subpopulation View
                          • New Subpopulation View (2)
                          • Detailed Look at PCA
                          • PCA Rediscovery ndash Renaming
                          • PCA Rediscovery ndash Renaming (2)
                          • PCA Rediscovery ndash Renaming (3)
                          • PCA Rediscovery ndash Renaming (4)
                          • PCA Rediscovery ndash Renaming (5)
                          • An Interesting Historical Note
                          • An Interesting Historical Note (2)
                          • Detailed Look at PCA (2)
                          • Detailed Look at PCA (3)
                          • Review of Linear Algebra
                          • Review of Linear Algebra (2)
                          • Review of Linear Algebra (3)
                          • Review of Linear Algebra (Cont)
                          • Review of Linear Algebra (Cont) (2)
                          • Review of Linear Algebra (Cont) (3)
                          • Review of Linear Algebra (Cont) (4)
                          • Review of Linear Algebra (Cont) (5)
                          • Review of Linear Algebra (Cont) (6)
                          • Review of Linear Algebra (Cont) (7)
                          • Review of Linear Algebra (Cont) (8)
                          • Review of Linear Algebra (Cont) (9)
                          • Review of Linear Algebra (Cont) (10)
                          • Review of Linear Algebra (Cont) (11)
                          • Review of Linear Algebra (Cont) (12)
                          • Review of Linear Algebra (Cont) (13)
                          • Review of Linear Algebra (Cont) (14)
                          • Review of Linear Algebra (Cont) (15)
                          • Review of Linear Algebra (Cont) (16)
                          • Review of Linear Algebra (Cont) (17)
                          • Review of Linear Algebra (Cont) (18)
                          • Review of Linear Algebra (Cont) (19)
                          • Review of Linear Algebra (Cont) (20)
                          • Review of Linear Algebra (Cont) (21)
                          • Review of Linear Algebra (Cont) (22)
                          • Review of Linear Algebra (Cont) (23)
                          • Review of Linear Algebra (Cont) (24)
                          • Review of Linear Algebra (Cont) (25)
                          • Review of Linear Algebra (Cont) (26)
                          • Review of Linear Algebra (Cont) (27)
                          • Review of Linear Algebra (Cont) (28)
                          • Review of Linear Algebra (Cont) (29)
                          • Review of Linear Algebra (Cont) (30)
                          • Review of Linear Algebra (Cont) (31)
                          • Review of Linear Algebra (Cont) (32)
                          • Review of Linear Algebra (Cont) (33)
                          • Review of Linear Algebra (Cont) (34)
                          • Review of Linear Algebra (Cont) (35)
                          • Review of Linear Algebra (Cont) (36)
                          • Review of Linear Algebra (Cont) (37)
                          • Review of Linear Algebra (Cont) (38)
                          • Review of Linear Algebra (Cont) (39)
                          • Review of Linear Algebra (Cont) (40)
                          • Review of Linear Algebra (Cont) (41)
                          • Review of Linear Algebra (Cont) (42)
                          • Review of Linear Algebra (Cont) (43)
                          • Review of Linear Algebra (Cont) (44)
                          • Review of Linear Algebra (Cont) (45)
                          • Review of Linear Algebra (Cont) (46)
                          • Review of Linear Algebra (Cont) (47)
                          • Review of Linear Algebra (Cont) (48)
                          • Review of Linear Algebra (Cont) (49)
                          • Review of Linear Algebra (Cont) (50)
                          • Review of Linear Algebra (Cont) (51)
                          • Review of Linear Algebra (Cont) (52)
                          • Review of Linear Algebra (Cont) (53)
                          • Review of Linear Algebra (Cont) (54)
                          • Review of Linear Algebra (Cont) (55)
                          • Review of Linear Algebra (Cont) (56)
                          • Review of Linear Algebra (Cont) (57)
                          • Review of Linear Algebra (Cont) (58)
                          • Review of Linear Algebra (Cont) (59)
                          • Review of Linear Algebra (Cont) (60)
                          • Review of Linear Algebra (Cont) (61)
                          • Review of Linear Algebra (Cont) (62)
                          • Review of Linear Algebra (Cont) (63)
                          • Review of Linear Algebra (Cont) (64)
                          • Review of Linear Algebra (Cont) (65)
                          • Review of Linear Algebra (Cont) (66)
                          • Review of Linear Algebra (Cont) (67)
                          • Review of Linear Algebra (Cont) (68)
                          • Review of Linear Algebra (Cont) (69)
                          • Review of Linear Algebra (Cont) (70)
                          • Review of Linear Algebra (Cont) (71)
                          • Review of Linear Algebra (Cont) (72)
                          • Review of Linear Algebra (Cont) (73)
                          • Review of Linear Algebra (Cont) (74)
                          • Review of Linear Algebra (Cont) (75)
                          • Review of Linear Algebra (Cont) (76)
                          • Review of Linear Algebra (Cont) (77)
                          • Review of Linear Algebra (Cont) (78)
                          • Review of Linear Algebra (Cont) (79)
                          • Review of Linear Algebra (Cont) (80)
                          • Review of Linear Algebra (Cont) (81)

                            PCA to find clustersSiZer analysis of Mass Flux PC1

                            PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                            PCA to find clustersSiZer analysis of Mass Flux PC1

                            Also in Curvature

                            PCA to find clustersSiZer analysis of Mass Flux PC1

                            And in Other Comprsquos

                            PCA to find clustersSiZer analysis of Mass Flux PC1

                            Conclusion

                            bull Found 3 significant clusters

                            bull Correspond to 3 known ldquocloud typesrdquo

                            bull Worth deeper investigation

                            Recall Yeast Cell Cycle Data

                            bull ldquoGene Expressionrdquo ndash Micro-array data

                            bull Data (after major preprocessing) Expression ldquolevelrdquo of

                            bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                            10s)bull Interesting statistical issue

                            High Dimension Low Sample Size data

                            (HDLSS)

                            Yeast Cell Cycle Data FDA View

                            Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                            Yeast Cell Cycle Data FDA View

                            Periodic genes

                            Naiumlve

                            approach

                            Simple PCA

                            Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                            ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                            ldquovariationrdquobull PCA finds ldquodirections of maximal

                            variationrdquobull Often but not always same as

                            ldquointeresting directionsrdquobull Here need better approach to study

                            periodicities

                            Yeast Cell Cycles Freq 2 Proj

                            PCA on

                            Freq 2

                            Periodic

                            Component

                            Of Data

                            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                            Approach from Zhao Marron amp Wells (2004)

                            Frequency 2 Analysis

                            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                            classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                            notable differencesbull Now try to improve ldquophase classificationrdquo

                            Yeast Cell CycleRevisit ldquophase classificationrdquo

                            approachbull Use outer 200 genes

                            (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                            (finds significant bumps etc in histogram)

                            bull Carefully redrew boundariesbull Check by studying kde angles

                            SiZer Study of Distrsquon of Angles

                            Reclassification of Major Genes

                            Compare to Previous Classifrsquon

                            New Subpopulation View

                            New Subpopulation View

                            NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                            Detailed Look at PCA

                            Now Study ldquoFolklorerdquo More Carefully

                            bull BackGround

                            bull History

                            bull Underpinnings

                            (Mathematical amp Computational)

                            Good Overall Reference Jolliffe (2002)

                            PCA Rediscovery ndash Renaming

                            Statistics Principal Component Analysis (PCA)

                            PCA Rediscovery ndash Renaming

                            Statistics Principal Component Analysis (PCA)

                            Social Sciences Factor Analysis (PCA is a subset)

                            PCA Rediscovery ndash Renaming

                            Statistics Principal Component Analysis (PCA)

                            Social Sciences Factor Analysis (PCA is a subset)

                            Probability Electrical EngKarhunen ndash Loeve expansion

                            PCA Rediscovery ndash Renaming

                            Statistics Principal Component Analysis (PCA)

                            Social Sciences Factor Analysis (PCA is a subset)

                            Probability Electrical EngKarhunen ndash Loeve expansion

                            Applied MathematicsProper Orthogonal Decomposition (POD)

                            PCA Rediscovery ndash Renaming

                            Statistics Principal Component Analysis (PCA)

                            Social Sciences Factor Analysis (PCA is a subset)

                            Probability Electrical EngKarhunen ndash Loeve expansion

                            Applied MathematicsProper Orthogonal Decomposition (POD)

                            Geo-Sciences Empirical Orthogonal Functions (EOF)

                            An Interesting Historical Note

                            The 1st () application of PCA to Functional

                            Data Analysis

                            An Interesting Historical Note

                            The 1st () application of PCA to Functional

                            Data Analysis

                            Rao (1958)

                            1st Paper with ldquoCurves as Data Objectsrdquo

                            viewpoint

                            Detailed Look at PCA

                            Three Important (amp Interesting) Viewpoints

                            1 Mathematics

                            2 Numerics

                            3 Statistics

                            Goal Study Interrelationships

                            Detailed Look at PCA

                            Three Important (amp Interesting) Viewpoints

                            1 Mathematics

                            2 Numerics

                            3 Statistics

                            1st Review Linear Alg and Multivar Prob

                            Review of Linear Algebra

                            Vector Space

                            bull set of ldquovectorsrdquo

                            bull and ldquoscalarsrdquo (coefficients)

                            xa

                            Review of Linear Algebra

                            Vector Space

                            bull set of ldquovectorsrdquo

                            bull and ldquoscalarsrdquo (coefficients)

                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                            ( in space)

                            xa

                            i

                            ii xa

                            Review of Linear Algebra

                            Vector Space

                            bull set of ldquovectorsrdquo

                            bull and ldquoscalarsrdquo (coefficients)

                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                            ( in space)

                            eg

                            ldquo dim Euclidrsquon spacerdquo

                            xa

                            i

                            ii xa

                            d

                            d

                            d xx

                            x

                            x

                            x 1

                            1

                            d

                            Review of Linear Algebra (Cont)

                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                            Review of Linear Algebra (Cont)

                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                            Review of Linear Algebra (Cont)

                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                            Note Planes not Through the Origin

                            are not Subspaces

                            (Do not Contain )00 x

                            Review of Linear Algebra (Cont)

                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                            (all Linear Combos of them =

                            = Containing Hyperplane

                            through Origin)

                            Review of Linear Algebra (Cont)

                            Basis of Subspace Set of Vectors that

                            bull Span ie Everything is a Lin Com of them

                            bull are Linearly Indeprsquot ie Lin Com is Unique

                            Review of Linear Algebra (Cont)

                            Basis of Subspace Set of Vectors that

                            bull Span ie Everything is a Lin Com of them

                            bull are Linearly Indeprsquot ie Lin Com is Unique

                            bull eg ldquoUnit Vector Basisrdquo

                            bull Since

                            d

                            1

                            0

                            0

                            0

                            1

                            0

                            0

                            0

                            1

                            1

                            0

                            0

                            0

                            1

                            0

                            0

                            0

                            1

                            212

                            1

                            d

                            d

                            xxx

                            x

                            x

                            x

                            Review of Linear Algebra (Cont)

                            Basis Matrix of subspace of

                            Given a basis

                            create matrix of columns

                            dnvv 1

                            nddnd

                            n

                            n

                            vv

                            vv

                            vvB

                            1

                            111

                            1

                            Review of Linear Algebra (Cont)

                            Then linear combo is a matrix multiplicatrsquon

                            where

                            n

                            iii aBva

                            1

                            na

                            a

                            a 1

                            Review of Linear Algebra (Cont)

                            Then linear combo is a matrix multiplicatrsquon

                            where

                            Note Right Multiplication Gives

                            Linear Combination of Column Vectors

                            n

                            iii aBva

                            1

                            na

                            a

                            a 1

                            Review of Linear Algebra (Cont)

                            Then linear combo is a matrix multiplicatrsquon

                            where

                            Check sizes

                            n

                            iii aBva

                            1

                            na

                            a

                            a 1

                            )1()(1 nndd

                            Review of Linear Algebra (Cont)

                            Aside on Matrix Multiplication (linear transformatrsquon)

                            For matrices

                            Define the Matrix Product

                            mkk

                            m

                            aa

                            aa

                            A

                            1

                            111

                            nmm

                            n

                            bb

                            bb

                            B

                            1

                            111

                            m

                            iniik

                            m

                            iiik

                            m

                            inii

                            m

                            iii

                            baba

                            baba

                            AB

                            1

                            11

                            11

                            111

                            Review of Linear Algebra (Cont)

                            Aside on Matrix Multiplication (linear transformatrsquon)

                            For matrices

                            Define the Matrix Product

                            (Inner Products of Rows With Columns )

                            mkk

                            m

                            aa

                            aa

                            A

                            1

                            111

                            nmm

                            n

                            bb

                            bb

                            B

                            1

                            111

                            m

                            iniik

                            m

                            iiik

                            m

                            inii

                            m

                            iii

                            baba

                            baba

                            AB

                            1

                            11

                            11

                            111

                            A B

                            Review of Linear Algebra (Cont)

                            Aside on Matrix Multiplication (linear transformatrsquon)

                            For matrices

                            Define the Matrix Product

                            (Inner Products of Rows With Columns )

                            (Composition of Linear Transformations)

                            mkk

                            m

                            aa

                            aa

                            A

                            1

                            111

                            nmm

                            n

                            bb

                            bb

                            B

                            1

                            111

                            m

                            iniik

                            m

                            iiik

                            m

                            inii

                            m

                            iii

                            baba

                            baba

                            AB

                            1

                            11

                            11

                            111

                            A B

                            Review of Linear Algebra (Cont)

                            Aside on Matrix Multiplication (linear transformatrsquon)

                            For matrices

                            Define the Matrix Product

                            (Inner Products of Rows With Columns )

                            (Composition of Linear Transformations)

                            Often Useful to Check Sizes

                            mkk

                            m

                            aa

                            aa

                            A

                            1

                            111

                            nmm

                            n

                            bb

                            bb

                            B

                            1

                            111

                            m

                            iniik

                            m

                            iiik

                            m

                            inii

                            m

                            iii

                            baba

                            baba

                            AB

                            1

                            11

                            11

                            111

                            nmmknk

                            A B

                            Review of Linear Algebra (Cont)

                            Aside on Matrix Multiplication (linear transformatrsquon)

                            For matrices

                            Define the Matrix Product

                            (Inner Products of Rows With Columns )

                            (Composition of Linear Transformations)

                            Often Useful to Check Sizes

                            mkk

                            m

                            aa

                            aa

                            A

                            1

                            111

                            nmm

                            n

                            bb

                            bb

                            B

                            1

                            111

                            m

                            iniik

                            m

                            iiik

                            m

                            inii

                            m

                            iii

                            baba

                            baba

                            AB

                            1

                            11

                            11

                            111

                            nmmknk

                            A B

                            Review of Linear Algebra (Cont)

                            Matrix Trace

                            bull For a Square Matrix

                            bull Define

                            mmm

                            m

                            aa

                            aa

                            A

                            1

                            111

                            m

                            iiiaAtr

                            1)(

                            Review of Linear Algebra (Cont)

                            Matrix Trace

                            bull For a Square Matrix

                            bull Define

                            bull Trace Commutes with Matrix Multiplication

                            mmm

                            m

                            aa

                            aa

                            A

                            1

                            111

                            m

                            iiiaAtr

                            1)(

                            BAtrABtr

                            Review of Linear Algebra (Cont)

                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                            bull Number of Elements in a Basis (Unique)

                            Review of Linear Algebra (Cont)

                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                            bull Number of Elements in a Basis (Unique)

                            bull (Use Basis Above)

                            bull eg dim of a line is 1

                            bull eg dim of a plane is 2

                            dd dim

                            Review of Linear Algebra (Cont)

                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                            bull Number of Elements in a Basis (Unique)

                            bull (Use Basis Above)

                            bull eg dim of a line is 1

                            bull eg dim of a plane is 2

                            bull Dimension is ldquoDegrees of Freedomrdquo

                            (in Statistical Uses eg ANOVA)

                            dd dim

                            Review of Linear Algebra (Cont)

                            Norm of a Vector

                            bull in d 21

                            21

                            1

                            2 xxxx td

                            jj

                            Review of Linear Algebra (Cont)

                            Norm of a Vector

                            bull in

                            bull Idea length of the vector

                            d 2121

                            1

                            2 xxxx td

                            jj

                            Review of Linear Algebra (Cont)

                            Norm of a Vector

                            bull in

                            bull Idea length of the vector

                            bull Note strange properties for high

                            eg ldquolength of diagonal of unit cuberdquo =

                            d 2121

                            1

                            2 xxxx td

                            jj

                            d

                            d

                            Review of Linear Algebra (Cont)

                            Norm of a Vector (cont)

                            bull Length Normalized Vector

                            (has Length 1 thus on Surf of Unit Sphere

                            amp is a Direction Vector)

                            x

                            x

                            Review of Linear Algebra (Cont)

                            Norm of a Vector (cont)

                            bull Length Normalized Vector

                            (has Length 1 thus on Surf of Unit Sphere

                            amp is a Direction Vector)

                            bull Define Distance as

                            x

                            x

                            yxyxyxyxd t

                            Review of Linear Algebra (Cont)

                            Inner (Dot Scalar) Product

                            bull for Vectors and

                            yxyxyx td

                            jjj

                            1

                            x y

                            Review of Linear Algebra (Cont)

                            Inner (Dot Scalar) Product

                            bull for Vectors and

                            bull Related to Norm via

                            yxyxyx td

                            jjj

                            1

                            x y

                            21 xxx

                            Review of Linear Algebra (Cont)

                            Inner (Dot Scalar) Product (cont)

                            bull measures ldquoangle between and rdquo as

                            yyxx

                            yx

                            yx

                            yxyxangle

                            tt

                            t

                            11 cos

                            cos

                            x y

                            Review of Linear Algebra (Cont)

                            Inner (Dot Scalar) Product (cont)

                            bull measures ldquoangle between and rdquo as

                            bull key to Orthogonality ie Perpendiculrsquoty

                            if and only if

                            yyxx

                            yx

                            yx

                            yxyxangle

                            tt

                            t

                            11 cos

                            cos

                            x y

                            yx 0 yx

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis

                            bull All Orthogonal to each other

                            ie for

                            bull All have Length 1

                            ie for

                            nvv 1

                            1 ii vv

                            0 ii vv ii

                            ni 1

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis (cont)

                            bull Spectral Representation

                            where

                            nvv 1

                            n

                            iii vax

                            1

                            ii vxa

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis (cont)

                            bull Spectral Representation

                            where

                            (Coefficient is Inner Product Cool Notation)

                            nvv 1

                            n

                            iii vax

                            1

                            ii vxa

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis (cont)

                            bull Spectral Representation

                            where

                            Check

                            nvv 1

                            n

                            iii vax

                            1

                            ii vxa

                            iii

                            n

                            iii

                            n

                            iiii avvavvavx

                            1

                            1

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis (cont)

                            bull Spectral Representation

                            where

                            Check

                            bull Matrix Notation where ie

                            For the Basis Matrix

                            nvv 1

                            n

                            iii vax

                            1

                            ii vxa

                            iii

                            n

                            iii

                            n

                            iiii avvavvavx

                            1

                            1

                            aBx Bxa tt xBa t

                            nvvB 1

                            Review of Linear Algebra (Cont)

                            Orthonormal Basis (cont)

                            bull Spectral Representation

                            where

                            Check

                            bull Matrix Notation where ie

                            is called transform of

                            (eg Fourier or Wavelet)

                            nvv 1

                            n

                            iii vax

                            1

                            ii vxa

                            iii

                            n

                            iii

                            n

                            iiii avvavvavx

                            1

                            1

                            aBx Bxa tt xBa t

                            xa

                            Review of Linear Algebra (Cont)

                            Parseval identity for

                            in subsp genrsquod by o n basis

                            x

                            nvv 1

                            2

                            1

                            22

                            1

                            2 aavxx

                            n

                            ii

                            n

                            ii

                            Review of Linear Algebra (Cont)

                            Parseval identity for

                            in subsp genrsquod by o n basis

                            bull Pythagorean theorem

                            bull ldquoDecomposition of Energyrdquo

                            bull ANOVA - sums of squares

                            x

                            nvv 1

                            2

                            1

                            22

                            1

                            2 aavxx

                            n

                            ii

                            n

                            ii

                            Review of Linear Algebra (Cont)

                            Parseval identity for

                            in subsp genrsquod by o n basis

                            bull Pythagorean theorem

                            bull ldquoDecomposition of Energyrdquo

                            bull ANOVA - sums of squares

                            bull Transform has same length as

                            ie ldquorotation in rdquo

                            x

                            nvv 1

                            2

                            1

                            22

                            1

                            2 aavxx

                            n

                            ii

                            n

                            ii

                            a xd

                            Projection of a Vector onto a Subspace

                            bull Idea Member of that is Closest to

                            (ie ldquoBest Approxrsquonrdquo)

                            Review of Linear Algebra (Cont)x

                            xV

                            V

                            Projection of a Vector onto a Subspace

                            bull Idea Member of that is Closest to

                            (ie ldquoBest Approxrsquonrdquo)

                            bull Find that Solves

                            (ldquoLeast Squaresrdquo)

                            Review of Linear Algebra (Cont)x

                            xV

                            V

                            VxPV vxVv

                            min

                            Projection of a Vector onto a Subspace

                            bull Idea Member of that is Closest to

                            (ie ldquoBest Approxrsquonrdquo)

                            bull Find that Solves

                            (ldquoLeast Squaresrdquo)

                            bull For Inner Product (Hilbert) Space

                            Exists and is Unique

                            Review of Linear Algebra (Cont)x

                            xV

                            V

                            VxPV vxVv

                            min

                            xPV

                            Projection of a Vector onto a Subspace (cont)

                            bull General Solution in for Basis Matrix

                            Review of Linear Algebra (Cont)

                            d VB

                            xBBBBxP tVV

                            tVVV

                            1

                            Projection of a Vector onto a Subspace (cont)

                            bull General Solution in for Basis Matrix

                            bull So Projrsquon Operator is Matrix Multrsquon

                            (thus projection is another linear operation)

                            Review of Linear Algebra (Cont)

                            d VB

                            xBBBBxP tVV

                            tVVV

                            1

                            tVV

                            tVVV BBBBP

                            1

                            Projection of a Vector onto a Subspace (cont)

                            bull General Solution in for Basis Matrix

                            bull So Projrsquon Operator is Matrix Multrsquon

                            (thus projection is another linear operation)

                            (note same operation underlies least squares)

                            Review of Linear Algebra (Cont)

                            d VB

                            xBBBBxP tVV

                            tVVV

                            1

                            tVV

                            tVVV BBBBP

                            1

                            Review of Linear Algebra (Cont)

                            Projection using Orthonormal Basis

                            bull Basis Matrix is Orthonormal

                            nnVtV IBB

                            10

                            01

                            1

                            111

                            1

                            1

                            nnn

                            n

                            ntn

                            t

                            vvvv

                            vvvv

                            vv

                            v

                            v

                            nvv 1

                            Review of Linear Algebra (Cont)

                            Projection using Orthonormal Basis

                            bull Basis Matrix is Orthonormal

                            bull So =

                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                            (Recall Right Multrsquon)

                            nnVtV IBB

                            10

                            01

                            1

                            111

                            1

                            1

                            nnn

                            n

                            ntn

                            t

                            vvvv

                            vvvv

                            vv

                            v

                            v

                            xBBxP tVVV

                            x V

                            nvv 1

                            Review of Linear Algebra (Cont)

                            Projection using Orthonormal Basis (cont)

                            bull For Orthogonal Complement

                            and

                            V

                            xPxPx VV 222xPxPx VV

                            Review of Linear Algebra (Cont)

                            Projection using Orthonormal Basis (cont)

                            bull For Orthogonal Complement

                            and

                            bull Parseval Inequality

                            V

                            xPxPx VV 222xPxPx VV

                            2

                            1

                            22

                            1

                            22 aavxxxP

                            n

                            ii

                            n

                            iiV

                            Review of Linear Algebra (Cont)

                            (Real) Unitary Matrices with

                            bull Orthonormal Basis Matrix

                            (So All of Above Applies)

                            ddU IUU t

                            Review of Linear Algebra (Cont)

                            (Real) Unitary Matrices with

                            bull Orthonormal Basis Matrix

                            (So All of Above Applies)

                            bull Note Transformrsquon is Distance Preserving

                            ddU IUU t

                            yxdyxyxyxUyUxUdn

                            i ii 2

                            1

                            Review of Linear Algebra (Cont)

                            (Real) Unitary Matrices with

                            bull Orthonormal Basis Matrix

                            (So All of Above Applies)

                            bull Note Transformrsquon is Distance Preserving

                            bull Lin Trans (Mult by ) is ~ Rotation

                            bull But also Includes ldquoMirror Imagesrdquo

                            ddU IUU t

                            yxdyxyxyxUyUxUdn

                            i ii 2

                            1

                            U

                            Review of Linear Algebra (Cont)

                            Singular Value Decomposition (SVD)

                            For a Matrix

                            Find

                            ndX

                            Review of Linear Algebra (Cont)

                            Singular Value Decomposition (SVD)

                            For a Matrix

                            Find a Diagonal Matrix

                            with Entries

                            called Singular Values

                            ndX

                            ndS

                            )min(1 ndss

                            Review of Linear Algebra (Cont)

                            Singular Value Decomposition (SVD)

                            For a Matrix

                            Find a Diagonal Matrix

                            with Entries

                            called Singular Values

                            And Unitary (Rotation) Matrices

                            (recall )

                            ndX

                            ndS

                            )min(1 ndss

                            ddU nnV

                            IVVUU tt

                            Review of Linear Algebra (Cont)

                            Singular Value Decomposition (SVD)

                            For a Matrix

                            Find a Diagonal Matrix

                            with Entries

                            called Singular Values

                            And Unitary (Rotation) Matrices

                            (recall )

                            So That

                            ndX

                            ndS

                            )min(1 ndss

                            ddU nnV

                            IVVUU tt tUSVX

                            Review of Linear Algebra (Cont)

                            Intuition behind Singular Value Decomposition

                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                            vVSUvVSUvX tt

                            Review of Linear Algebra (Cont)

                            Intuition behind Singular Value Decomposition

                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                            bull First rotate

                            X

                            vVSUvVSUvX tt

                            Review of Linear Algebra (Cont)

                            Intuition behind Singular Value Decomposition

                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                            bull First rotate

                            bull Second rescale coordinate axes (by )

                            X

                            vVSUvVSUvX tt

                            is

                            Review of Linear Algebra (Cont)

                            Intuition behind Singular Value Decomposition

                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                            bull First rotate

                            bull Second rescale coordinate axes (by )

                            bull Third rotate again

                            X

                            vVSUvVSUvX tt

                            is

                            Review of Linear Algebra (Cont)

                            Intuition behind Singular Value Decomposition

                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                            bull First rotate

                            bull Second rescale coordinate axes (by )

                            bull Third rotate again

                            bull ie have diagonalized the transformation

                            X

                            vVSUvVSUvX tt

                            is

                            SVD Compact Representation

                            Useful Labeling

                            Singular Values in Increasing Order

                            Review of Linear Algebra (Cont)

                            )min(1 dnss

                            SVD Compact Representation

                            Useful Labeling

                            Singular Values in Increasing Order

                            Note singular values = 0 can be omitted

                            (Since do ldquo0-Stretchingrdquo)

                            Review of Linear Algebra (Cont)

                            )min(1 dnss

                            Review of Linear Algebra (Cont)

                            r

                            SVD Compact Representation

                            Useful Labeling

                            Singular Values in Increasing Order

                            Note singular values = 0 can be omitted

                            Let = of positive singular values

                            )min(1 dnss

                            Review of Linear Algebra (Cont)

                            r

                            SVD Compact Representation

                            Useful Labeling

                            Singular Values in Increasing Order

                            Note singular values = 0 can be omitted

                            Let = of positive singular values

                            Then

                            Where are truncations of

                            trnrrrd VSUX

                            VSU

                            )min(1 dnss

                            Review of Linear Algebra (Cont)

                            SVD Full Representation

                            =

                            Graphics Display Assumes

                            ndX ddU ndS nn

                            tV

                            nd

                            Review of Linear Algebra (Cont)

                            SVD Full Representation

                            =

                            Full Rank Basis Matrix

                            ndX ddU ndS nn

                            tV

                            Review of Linear Algebra (Cont)

                            SVD Full Representation

                            =

                            Full Rank Basis Matrix

                            All 0s in Bottom

                            ndX ddU ndS nn

                            tV

                            Review of Linear Algebra (Cont)

                            SVD Reduced Representation

                            =

                            These Columns Get 0ed Out

                            ndX ddU nnS nn

                            tV

                            nnd 0

                            Review of Linear Algebra (Cont)

                            SVD Reduced Representation

                            =ndX ndU nnS nn

                            tV

                            Review of Linear Algebra (Cont)

                            SVD Reduced Representation

                            =

                            Also Some of These May be 0

                            ndX ndU nnS nn

                            tV

                            Review of Linear Algebra (Cont)

                            SVD Compact Representation

                            = ndX rdU

                            rrS nrtV

                            0

                            Review of Linear Algebra (Cont)

                            SVD Compact Representation

                            =

                            These Get 0ed Out

                            ndX rdU

                            rrS nrtV

                            0

                            Review of Linear Algebra (Cont)

                            SVD Compact Representation

                            = ndX rdU

                            rrS nrtV

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition

                            For a (Symmetric) Square Matrix

                            Find

                            ddX

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition

                            For a (Symmetric) Square Matrix

                            Find a Diagonal Matrix

                            Called Eigenvalues

                            Convenient Ordering

                            ddX

                            d

                            D

                            0

                            01

                            n 1

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition

                            For a (Symmetric) Square Matrix

                            Find a Diagonal Matrix

                            And an Orthonormal Matrix

                            (ie )

                            ddX

                            d

                            D

                            0

                            01

                            ddB

                            ddtt IBBBB

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition

                            For a (Symmetric) Square Matrix

                            Find a Diagonal Matrix

                            And an Orthonormal Matrix

                            (ie )

                            So that ie

                            ddX

                            d

                            D

                            0

                            01

                            ddB

                            ddtt IBBBB

                            DBBX tBDBX

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                            (looks similar)

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                            Complex (uses )

                            VU

                            1i

                            Review of Linear Algebra (Cont)

                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                            VU

                            X

                            BVU

                            1i

                            • Participant Presentations
                            • PCA to find clusters
                            • Statistical Smoothing
                            • Kernel Density Estimation
                            • Scatterplot Smoothing
                            • Statistical Smoothing (2)
                            • SiZer Background
                            • SiZer Background (2)
                            • SiZer Background (3)
                            • SiZer Background (4)
                            • SiZer Background (5)
                            • SiZer Overview
                            • PCA to find clusters (2)
                            • PCA to find clusters (3)
                            • PCA to find clusters (4)
                            • PCA to find clusters (5)
                            • PCA to find clusters (6)
                            • PCA to find clusters (7)
                            • Recall Yeast Cell Cycle Data
                            • Yeast Cell Cycle Data FDA View
                            • Yeast Cell Cycle Data FDA View (2)
                            • Yeast Cell Cycle Data FDA View (3)
                            • Yeast Cell Cycles Freq 2 Proj
                            • Frequency 2 Analysis
                            • Frequency 2 Analysis (2)
                            • Frequency 2 Analysis (3)
                            • Yeast Cell Cycle
                            • SiZer Study of Distrsquon of Angles
                            • Reclassification of Major Genes
                            • Compare to Previous Classifrsquon
                            • New Subpopulation View
                            • New Subpopulation View (2)
                            • Detailed Look at PCA
                            • PCA Rediscovery ndash Renaming
                            • PCA Rediscovery ndash Renaming (2)
                            • PCA Rediscovery ndash Renaming (3)
                            • PCA Rediscovery ndash Renaming (4)
                            • PCA Rediscovery ndash Renaming (5)
                            • An Interesting Historical Note
                            • An Interesting Historical Note (2)
                            • Detailed Look at PCA (2)
                            • Detailed Look at PCA (3)
                            • Review of Linear Algebra
                            • Review of Linear Algebra (2)
                            • Review of Linear Algebra (3)
                            • Review of Linear Algebra (Cont)
                            • Review of Linear Algebra (Cont) (2)
                            • Review of Linear Algebra (Cont) (3)
                            • Review of Linear Algebra (Cont) (4)
                            • Review of Linear Algebra (Cont) (5)
                            • Review of Linear Algebra (Cont) (6)
                            • Review of Linear Algebra (Cont) (7)
                            • Review of Linear Algebra (Cont) (8)
                            • Review of Linear Algebra (Cont) (9)
                            • Review of Linear Algebra (Cont) (10)
                            • Review of Linear Algebra (Cont) (11)
                            • Review of Linear Algebra (Cont) (12)
                            • Review of Linear Algebra (Cont) (13)
                            • Review of Linear Algebra (Cont) (14)
                            • Review of Linear Algebra (Cont) (15)
                            • Review of Linear Algebra (Cont) (16)
                            • Review of Linear Algebra (Cont) (17)
                            • Review of Linear Algebra (Cont) (18)
                            • Review of Linear Algebra (Cont) (19)
                            • Review of Linear Algebra (Cont) (20)
                            • Review of Linear Algebra (Cont) (21)
                            • Review of Linear Algebra (Cont) (22)
                            • Review of Linear Algebra (Cont) (23)
                            • Review of Linear Algebra (Cont) (24)
                            • Review of Linear Algebra (Cont) (25)
                            • Review of Linear Algebra (Cont) (26)
                            • Review of Linear Algebra (Cont) (27)
                            • Review of Linear Algebra (Cont) (28)
                            • Review of Linear Algebra (Cont) (29)
                            • Review of Linear Algebra (Cont) (30)
                            • Review of Linear Algebra (Cont) (31)
                            • Review of Linear Algebra (Cont) (32)
                            • Review of Linear Algebra (Cont) (33)
                            • Review of Linear Algebra (Cont) (34)
                            • Review of Linear Algebra (Cont) (35)
                            • Review of Linear Algebra (Cont) (36)
                            • Review of Linear Algebra (Cont) (37)
                            • Review of Linear Algebra (Cont) (38)
                            • Review of Linear Algebra (Cont) (39)
                            • Review of Linear Algebra (Cont) (40)
                            • Review of Linear Algebra (Cont) (41)
                            • Review of Linear Algebra (Cont) (42)
                            • Review of Linear Algebra (Cont) (43)
                            • Review of Linear Algebra (Cont) (44)
                            • Review of Linear Algebra (Cont) (45)
                            • Review of Linear Algebra (Cont) (46)
                            • Review of Linear Algebra (Cont) (47)
                            • Review of Linear Algebra (Cont) (48)
                            • Review of Linear Algebra (Cont) (49)
                            • Review of Linear Algebra (Cont) (50)
                            • Review of Linear Algebra (Cont) (51)
                            • Review of Linear Algebra (Cont) (52)
                            • Review of Linear Algebra (Cont) (53)
                            • Review of Linear Algebra (Cont) (54)
                            • Review of Linear Algebra (Cont) (55)
                            • Review of Linear Algebra (Cont) (56)
                            • Review of Linear Algebra (Cont) (57)
                            • Review of Linear Algebra (Cont) (58)
                            • Review of Linear Algebra (Cont) (59)
                            • Review of Linear Algebra (Cont) (60)
                            • Review of Linear Algebra (Cont) (61)
                            • Review of Linear Algebra (Cont) (62)
                            • Review of Linear Algebra (Cont) (63)
                            • Review of Linear Algebra (Cont) (64)
                            • Review of Linear Algebra (Cont) (65)
                            • Review of Linear Algebra (Cont) (66)
                            • Review of Linear Algebra (Cont) (67)
                            • Review of Linear Algebra (Cont) (68)
                            • Review of Linear Algebra (Cont) (69)
                            • Review of Linear Algebra (Cont) (70)
                            • Review of Linear Algebra (Cont) (71)
                            • Review of Linear Algebra (Cont) (72)
                            • Review of Linear Algebra (Cont) (73)
                            • Review of Linear Algebra (Cont) (74)
                            • Review of Linear Algebra (Cont) (75)
                            • Review of Linear Algebra (Cont) (76)
                            • Review of Linear Algebra (Cont) (77)
                            • Review of Linear Algebra (Cont) (78)
                            • Review of Linear Algebra (Cont) (79)
                            • Review of Linear Algebra (Cont) (80)
                            • Review of Linear Algebra (Cont) (81)

                              PCA to find clustersSiZer analysis of Mass Flux PC1All 3Signifrsquot

                              PCA to find clustersSiZer analysis of Mass Flux PC1

                              Also in Curvature

                              PCA to find clustersSiZer analysis of Mass Flux PC1

                              And in Other Comprsquos

                              PCA to find clustersSiZer analysis of Mass Flux PC1

                              Conclusion

                              bull Found 3 significant clusters

                              bull Correspond to 3 known ldquocloud typesrdquo

                              bull Worth deeper investigation

                              Recall Yeast Cell Cycle Data

                              bull ldquoGene Expressionrdquo ndash Micro-array data

                              bull Data (after major preprocessing) Expression ldquolevelrdquo of

                              bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                              10s)bull Interesting statistical issue

                              High Dimension Low Sample Size data

                              (HDLSS)

                              Yeast Cell Cycle Data FDA View

                              Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                              Yeast Cell Cycle Data FDA View

                              Periodic genes

                              Naiumlve

                              approach

                              Simple PCA

                              Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                              ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                              ldquovariationrdquobull PCA finds ldquodirections of maximal

                              variationrdquobull Often but not always same as

                              ldquointeresting directionsrdquobull Here need better approach to study

                              periodicities

                              Yeast Cell Cycles Freq 2 Proj

                              PCA on

                              Freq 2

                              Periodic

                              Component

                              Of Data

                              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                              Approach from Zhao Marron amp Wells (2004)

                              Frequency 2 Analysis

                              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                              classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                              notable differencesbull Now try to improve ldquophase classificationrdquo

                              Yeast Cell CycleRevisit ldquophase classificationrdquo

                              approachbull Use outer 200 genes

                              (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                              (finds significant bumps etc in histogram)

                              bull Carefully redrew boundariesbull Check by studying kde angles

                              SiZer Study of Distrsquon of Angles

                              Reclassification of Major Genes

                              Compare to Previous Classifrsquon

                              New Subpopulation View

                              New Subpopulation View

                              NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                              Detailed Look at PCA

                              Now Study ldquoFolklorerdquo More Carefully

                              bull BackGround

                              bull History

                              bull Underpinnings

                              (Mathematical amp Computational)

                              Good Overall Reference Jolliffe (2002)

                              PCA Rediscovery ndash Renaming

                              Statistics Principal Component Analysis (PCA)

                              PCA Rediscovery ndash Renaming

                              Statistics Principal Component Analysis (PCA)

                              Social Sciences Factor Analysis (PCA is a subset)

                              PCA Rediscovery ndash Renaming

                              Statistics Principal Component Analysis (PCA)

                              Social Sciences Factor Analysis (PCA is a subset)

                              Probability Electrical EngKarhunen ndash Loeve expansion

                              PCA Rediscovery ndash Renaming

                              Statistics Principal Component Analysis (PCA)

                              Social Sciences Factor Analysis (PCA is a subset)

                              Probability Electrical EngKarhunen ndash Loeve expansion

                              Applied MathematicsProper Orthogonal Decomposition (POD)

                              PCA Rediscovery ndash Renaming

                              Statistics Principal Component Analysis (PCA)

                              Social Sciences Factor Analysis (PCA is a subset)

                              Probability Electrical EngKarhunen ndash Loeve expansion

                              Applied MathematicsProper Orthogonal Decomposition (POD)

                              Geo-Sciences Empirical Orthogonal Functions (EOF)

                              An Interesting Historical Note

                              The 1st () application of PCA to Functional

                              Data Analysis

                              An Interesting Historical Note

                              The 1st () application of PCA to Functional

                              Data Analysis

                              Rao (1958)

                              1st Paper with ldquoCurves as Data Objectsrdquo

                              viewpoint

                              Detailed Look at PCA

                              Three Important (amp Interesting) Viewpoints

                              1 Mathematics

                              2 Numerics

                              3 Statistics

                              Goal Study Interrelationships

                              Detailed Look at PCA

                              Three Important (amp Interesting) Viewpoints

                              1 Mathematics

                              2 Numerics

                              3 Statistics

                              1st Review Linear Alg and Multivar Prob

                              Review of Linear Algebra

                              Vector Space

                              bull set of ldquovectorsrdquo

                              bull and ldquoscalarsrdquo (coefficients)

                              xa

                              Review of Linear Algebra

                              Vector Space

                              bull set of ldquovectorsrdquo

                              bull and ldquoscalarsrdquo (coefficients)

                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                              ( in space)

                              xa

                              i

                              ii xa

                              Review of Linear Algebra

                              Vector Space

                              bull set of ldquovectorsrdquo

                              bull and ldquoscalarsrdquo (coefficients)

                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                              ( in space)

                              eg

                              ldquo dim Euclidrsquon spacerdquo

                              xa

                              i

                              ii xa

                              d

                              d

                              d xx

                              x

                              x

                              x 1

                              1

                              d

                              Review of Linear Algebra (Cont)

                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                              Review of Linear Algebra (Cont)

                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                              Review of Linear Algebra (Cont)

                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                              Note Planes not Through the Origin

                              are not Subspaces

                              (Do not Contain )00 x

                              Review of Linear Algebra (Cont)

                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                              (all Linear Combos of them =

                              = Containing Hyperplane

                              through Origin)

                              Review of Linear Algebra (Cont)

                              Basis of Subspace Set of Vectors that

                              bull Span ie Everything is a Lin Com of them

                              bull are Linearly Indeprsquot ie Lin Com is Unique

                              Review of Linear Algebra (Cont)

                              Basis of Subspace Set of Vectors that

                              bull Span ie Everything is a Lin Com of them

                              bull are Linearly Indeprsquot ie Lin Com is Unique

                              bull eg ldquoUnit Vector Basisrdquo

                              bull Since

                              d

                              1

                              0

                              0

                              0

                              1

                              0

                              0

                              0

                              1

                              1

                              0

                              0

                              0

                              1

                              0

                              0

                              0

                              1

                              212

                              1

                              d

                              d

                              xxx

                              x

                              x

                              x

                              Review of Linear Algebra (Cont)

                              Basis Matrix of subspace of

                              Given a basis

                              create matrix of columns

                              dnvv 1

                              nddnd

                              n

                              n

                              vv

                              vv

                              vvB

                              1

                              111

                              1

                              Review of Linear Algebra (Cont)

                              Then linear combo is a matrix multiplicatrsquon

                              where

                              n

                              iii aBva

                              1

                              na

                              a

                              a 1

                              Review of Linear Algebra (Cont)

                              Then linear combo is a matrix multiplicatrsquon

                              where

                              Note Right Multiplication Gives

                              Linear Combination of Column Vectors

                              n

                              iii aBva

                              1

                              na

                              a

                              a 1

                              Review of Linear Algebra (Cont)

                              Then linear combo is a matrix multiplicatrsquon

                              where

                              Check sizes

                              n

                              iii aBva

                              1

                              na

                              a

                              a 1

                              )1()(1 nndd

                              Review of Linear Algebra (Cont)

                              Aside on Matrix Multiplication (linear transformatrsquon)

                              For matrices

                              Define the Matrix Product

                              mkk

                              m

                              aa

                              aa

                              A

                              1

                              111

                              nmm

                              n

                              bb

                              bb

                              B

                              1

                              111

                              m

                              iniik

                              m

                              iiik

                              m

                              inii

                              m

                              iii

                              baba

                              baba

                              AB

                              1

                              11

                              11

                              111

                              Review of Linear Algebra (Cont)

                              Aside on Matrix Multiplication (linear transformatrsquon)

                              For matrices

                              Define the Matrix Product

                              (Inner Products of Rows With Columns )

                              mkk

                              m

                              aa

                              aa

                              A

                              1

                              111

                              nmm

                              n

                              bb

                              bb

                              B

                              1

                              111

                              m

                              iniik

                              m

                              iiik

                              m

                              inii

                              m

                              iii

                              baba

                              baba

                              AB

                              1

                              11

                              11

                              111

                              A B

                              Review of Linear Algebra (Cont)

                              Aside on Matrix Multiplication (linear transformatrsquon)

                              For matrices

                              Define the Matrix Product

                              (Inner Products of Rows With Columns )

                              (Composition of Linear Transformations)

                              mkk

                              m

                              aa

                              aa

                              A

                              1

                              111

                              nmm

                              n

                              bb

                              bb

                              B

                              1

                              111

                              m

                              iniik

                              m

                              iiik

                              m

                              inii

                              m

                              iii

                              baba

                              baba

                              AB

                              1

                              11

                              11

                              111

                              A B

                              Review of Linear Algebra (Cont)

                              Aside on Matrix Multiplication (linear transformatrsquon)

                              For matrices

                              Define the Matrix Product

                              (Inner Products of Rows With Columns )

                              (Composition of Linear Transformations)

                              Often Useful to Check Sizes

                              mkk

                              m

                              aa

                              aa

                              A

                              1

                              111

                              nmm

                              n

                              bb

                              bb

                              B

                              1

                              111

                              m

                              iniik

                              m

                              iiik

                              m

                              inii

                              m

                              iii

                              baba

                              baba

                              AB

                              1

                              11

                              11

                              111

                              nmmknk

                              A B

                              Review of Linear Algebra (Cont)

                              Aside on Matrix Multiplication (linear transformatrsquon)

                              For matrices

                              Define the Matrix Product

                              (Inner Products of Rows With Columns )

                              (Composition of Linear Transformations)

                              Often Useful to Check Sizes

                              mkk

                              m

                              aa

                              aa

                              A

                              1

                              111

                              nmm

                              n

                              bb

                              bb

                              B

                              1

                              111

                              m

                              iniik

                              m

                              iiik

                              m

                              inii

                              m

                              iii

                              baba

                              baba

                              AB

                              1

                              11

                              11

                              111

                              nmmknk

                              A B

                              Review of Linear Algebra (Cont)

                              Matrix Trace

                              bull For a Square Matrix

                              bull Define

                              mmm

                              m

                              aa

                              aa

                              A

                              1

                              111

                              m

                              iiiaAtr

                              1)(

                              Review of Linear Algebra (Cont)

                              Matrix Trace

                              bull For a Square Matrix

                              bull Define

                              bull Trace Commutes with Matrix Multiplication

                              mmm

                              m

                              aa

                              aa

                              A

                              1

                              111

                              m

                              iiiaAtr

                              1)(

                              BAtrABtr

                              Review of Linear Algebra (Cont)

                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                              bull Number of Elements in a Basis (Unique)

                              Review of Linear Algebra (Cont)

                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                              bull Number of Elements in a Basis (Unique)

                              bull (Use Basis Above)

                              bull eg dim of a line is 1

                              bull eg dim of a plane is 2

                              dd dim

                              Review of Linear Algebra (Cont)

                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                              bull Number of Elements in a Basis (Unique)

                              bull (Use Basis Above)

                              bull eg dim of a line is 1

                              bull eg dim of a plane is 2

                              bull Dimension is ldquoDegrees of Freedomrdquo

                              (in Statistical Uses eg ANOVA)

                              dd dim

                              Review of Linear Algebra (Cont)

                              Norm of a Vector

                              bull in d 21

                              21

                              1

                              2 xxxx td

                              jj

                              Review of Linear Algebra (Cont)

                              Norm of a Vector

                              bull in

                              bull Idea length of the vector

                              d 2121

                              1

                              2 xxxx td

                              jj

                              Review of Linear Algebra (Cont)

                              Norm of a Vector

                              bull in

                              bull Idea length of the vector

                              bull Note strange properties for high

                              eg ldquolength of diagonal of unit cuberdquo =

                              d 2121

                              1

                              2 xxxx td

                              jj

                              d

                              d

                              Review of Linear Algebra (Cont)

                              Norm of a Vector (cont)

                              bull Length Normalized Vector

                              (has Length 1 thus on Surf of Unit Sphere

                              amp is a Direction Vector)

                              x

                              x

                              Review of Linear Algebra (Cont)

                              Norm of a Vector (cont)

                              bull Length Normalized Vector

                              (has Length 1 thus on Surf of Unit Sphere

                              amp is a Direction Vector)

                              bull Define Distance as

                              x

                              x

                              yxyxyxyxd t

                              Review of Linear Algebra (Cont)

                              Inner (Dot Scalar) Product

                              bull for Vectors and

                              yxyxyx td

                              jjj

                              1

                              x y

                              Review of Linear Algebra (Cont)

                              Inner (Dot Scalar) Product

                              bull for Vectors and

                              bull Related to Norm via

                              yxyxyx td

                              jjj

                              1

                              x y

                              21 xxx

                              Review of Linear Algebra (Cont)

                              Inner (Dot Scalar) Product (cont)

                              bull measures ldquoangle between and rdquo as

                              yyxx

                              yx

                              yx

                              yxyxangle

                              tt

                              t

                              11 cos

                              cos

                              x y

                              Review of Linear Algebra (Cont)

                              Inner (Dot Scalar) Product (cont)

                              bull measures ldquoangle between and rdquo as

                              bull key to Orthogonality ie Perpendiculrsquoty

                              if and only if

                              yyxx

                              yx

                              yx

                              yxyxangle

                              tt

                              t

                              11 cos

                              cos

                              x y

                              yx 0 yx

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis

                              bull All Orthogonal to each other

                              ie for

                              bull All have Length 1

                              ie for

                              nvv 1

                              1 ii vv

                              0 ii vv ii

                              ni 1

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis (cont)

                              bull Spectral Representation

                              where

                              nvv 1

                              n

                              iii vax

                              1

                              ii vxa

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis (cont)

                              bull Spectral Representation

                              where

                              (Coefficient is Inner Product Cool Notation)

                              nvv 1

                              n

                              iii vax

                              1

                              ii vxa

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis (cont)

                              bull Spectral Representation

                              where

                              Check

                              nvv 1

                              n

                              iii vax

                              1

                              ii vxa

                              iii

                              n

                              iii

                              n

                              iiii avvavvavx

                              1

                              1

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis (cont)

                              bull Spectral Representation

                              where

                              Check

                              bull Matrix Notation where ie

                              For the Basis Matrix

                              nvv 1

                              n

                              iii vax

                              1

                              ii vxa

                              iii

                              n

                              iii

                              n

                              iiii avvavvavx

                              1

                              1

                              aBx Bxa tt xBa t

                              nvvB 1

                              Review of Linear Algebra (Cont)

                              Orthonormal Basis (cont)

                              bull Spectral Representation

                              where

                              Check

                              bull Matrix Notation where ie

                              is called transform of

                              (eg Fourier or Wavelet)

                              nvv 1

                              n

                              iii vax

                              1

                              ii vxa

                              iii

                              n

                              iii

                              n

                              iiii avvavvavx

                              1

                              1

                              aBx Bxa tt xBa t

                              xa

                              Review of Linear Algebra (Cont)

                              Parseval identity for

                              in subsp genrsquod by o n basis

                              x

                              nvv 1

                              2

                              1

                              22

                              1

                              2 aavxx

                              n

                              ii

                              n

                              ii

                              Review of Linear Algebra (Cont)

                              Parseval identity for

                              in subsp genrsquod by o n basis

                              bull Pythagorean theorem

                              bull ldquoDecomposition of Energyrdquo

                              bull ANOVA - sums of squares

                              x

                              nvv 1

                              2

                              1

                              22

                              1

                              2 aavxx

                              n

                              ii

                              n

                              ii

                              Review of Linear Algebra (Cont)

                              Parseval identity for

                              in subsp genrsquod by o n basis

                              bull Pythagorean theorem

                              bull ldquoDecomposition of Energyrdquo

                              bull ANOVA - sums of squares

                              bull Transform has same length as

                              ie ldquorotation in rdquo

                              x

                              nvv 1

                              2

                              1

                              22

                              1

                              2 aavxx

                              n

                              ii

                              n

                              ii

                              a xd

                              Projection of a Vector onto a Subspace

                              bull Idea Member of that is Closest to

                              (ie ldquoBest Approxrsquonrdquo)

                              Review of Linear Algebra (Cont)x

                              xV

                              V

                              Projection of a Vector onto a Subspace

                              bull Idea Member of that is Closest to

                              (ie ldquoBest Approxrsquonrdquo)

                              bull Find that Solves

                              (ldquoLeast Squaresrdquo)

                              Review of Linear Algebra (Cont)x

                              xV

                              V

                              VxPV vxVv

                              min

                              Projection of a Vector onto a Subspace

                              bull Idea Member of that is Closest to

                              (ie ldquoBest Approxrsquonrdquo)

                              bull Find that Solves

                              (ldquoLeast Squaresrdquo)

                              bull For Inner Product (Hilbert) Space

                              Exists and is Unique

                              Review of Linear Algebra (Cont)x

                              xV

                              V

                              VxPV vxVv

                              min

                              xPV

                              Projection of a Vector onto a Subspace (cont)

                              bull General Solution in for Basis Matrix

                              Review of Linear Algebra (Cont)

                              d VB

                              xBBBBxP tVV

                              tVVV

                              1

                              Projection of a Vector onto a Subspace (cont)

                              bull General Solution in for Basis Matrix

                              bull So Projrsquon Operator is Matrix Multrsquon

                              (thus projection is another linear operation)

                              Review of Linear Algebra (Cont)

                              d VB

                              xBBBBxP tVV

                              tVVV

                              1

                              tVV

                              tVVV BBBBP

                              1

                              Projection of a Vector onto a Subspace (cont)

                              bull General Solution in for Basis Matrix

                              bull So Projrsquon Operator is Matrix Multrsquon

                              (thus projection is another linear operation)

                              (note same operation underlies least squares)

                              Review of Linear Algebra (Cont)

                              d VB

                              xBBBBxP tVV

                              tVVV

                              1

                              tVV

                              tVVV BBBBP

                              1

                              Review of Linear Algebra (Cont)

                              Projection using Orthonormal Basis

                              bull Basis Matrix is Orthonormal

                              nnVtV IBB

                              10

                              01

                              1

                              111

                              1

                              1

                              nnn

                              n

                              ntn

                              t

                              vvvv

                              vvvv

                              vv

                              v

                              v

                              nvv 1

                              Review of Linear Algebra (Cont)

                              Projection using Orthonormal Basis

                              bull Basis Matrix is Orthonormal

                              bull So =

                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                              (Recall Right Multrsquon)

                              nnVtV IBB

                              10

                              01

                              1

                              111

                              1

                              1

                              nnn

                              n

                              ntn

                              t

                              vvvv

                              vvvv

                              vv

                              v

                              v

                              xBBxP tVVV

                              x V

                              nvv 1

                              Review of Linear Algebra (Cont)

                              Projection using Orthonormal Basis (cont)

                              bull For Orthogonal Complement

                              and

                              V

                              xPxPx VV 222xPxPx VV

                              Review of Linear Algebra (Cont)

                              Projection using Orthonormal Basis (cont)

                              bull For Orthogonal Complement

                              and

                              bull Parseval Inequality

                              V

                              xPxPx VV 222xPxPx VV

                              2

                              1

                              22

                              1

                              22 aavxxxP

                              n

                              ii

                              n

                              iiV

                              Review of Linear Algebra (Cont)

                              (Real) Unitary Matrices with

                              bull Orthonormal Basis Matrix

                              (So All of Above Applies)

                              ddU IUU t

                              Review of Linear Algebra (Cont)

                              (Real) Unitary Matrices with

                              bull Orthonormal Basis Matrix

                              (So All of Above Applies)

                              bull Note Transformrsquon is Distance Preserving

                              ddU IUU t

                              yxdyxyxyxUyUxUdn

                              i ii 2

                              1

                              Review of Linear Algebra (Cont)

                              (Real) Unitary Matrices with

                              bull Orthonormal Basis Matrix

                              (So All of Above Applies)

                              bull Note Transformrsquon is Distance Preserving

                              bull Lin Trans (Mult by ) is ~ Rotation

                              bull But also Includes ldquoMirror Imagesrdquo

                              ddU IUU t

                              yxdyxyxyxUyUxUdn

                              i ii 2

                              1

                              U

                              Review of Linear Algebra (Cont)

                              Singular Value Decomposition (SVD)

                              For a Matrix

                              Find

                              ndX

                              Review of Linear Algebra (Cont)

                              Singular Value Decomposition (SVD)

                              For a Matrix

                              Find a Diagonal Matrix

                              with Entries

                              called Singular Values

                              ndX

                              ndS

                              )min(1 ndss

                              Review of Linear Algebra (Cont)

                              Singular Value Decomposition (SVD)

                              For a Matrix

                              Find a Diagonal Matrix

                              with Entries

                              called Singular Values

                              And Unitary (Rotation) Matrices

                              (recall )

                              ndX

                              ndS

                              )min(1 ndss

                              ddU nnV

                              IVVUU tt

                              Review of Linear Algebra (Cont)

                              Singular Value Decomposition (SVD)

                              For a Matrix

                              Find a Diagonal Matrix

                              with Entries

                              called Singular Values

                              And Unitary (Rotation) Matrices

                              (recall )

                              So That

                              ndX

                              ndS

                              )min(1 ndss

                              ddU nnV

                              IVVUU tt tUSVX

                              Review of Linear Algebra (Cont)

                              Intuition behind Singular Value Decomposition

                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                              vVSUvVSUvX tt

                              Review of Linear Algebra (Cont)

                              Intuition behind Singular Value Decomposition

                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                              bull First rotate

                              X

                              vVSUvVSUvX tt

                              Review of Linear Algebra (Cont)

                              Intuition behind Singular Value Decomposition

                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                              bull First rotate

                              bull Second rescale coordinate axes (by )

                              X

                              vVSUvVSUvX tt

                              is

                              Review of Linear Algebra (Cont)

                              Intuition behind Singular Value Decomposition

                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                              bull First rotate

                              bull Second rescale coordinate axes (by )

                              bull Third rotate again

                              X

                              vVSUvVSUvX tt

                              is

                              Review of Linear Algebra (Cont)

                              Intuition behind Singular Value Decomposition

                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                              bull First rotate

                              bull Second rescale coordinate axes (by )

                              bull Third rotate again

                              bull ie have diagonalized the transformation

                              X

                              vVSUvVSUvX tt

                              is

                              SVD Compact Representation

                              Useful Labeling

                              Singular Values in Increasing Order

                              Review of Linear Algebra (Cont)

                              )min(1 dnss

                              SVD Compact Representation

                              Useful Labeling

                              Singular Values in Increasing Order

                              Note singular values = 0 can be omitted

                              (Since do ldquo0-Stretchingrdquo)

                              Review of Linear Algebra (Cont)

                              )min(1 dnss

                              Review of Linear Algebra (Cont)

                              r

                              SVD Compact Representation

                              Useful Labeling

                              Singular Values in Increasing Order

                              Note singular values = 0 can be omitted

                              Let = of positive singular values

                              )min(1 dnss

                              Review of Linear Algebra (Cont)

                              r

                              SVD Compact Representation

                              Useful Labeling

                              Singular Values in Increasing Order

                              Note singular values = 0 can be omitted

                              Let = of positive singular values

                              Then

                              Where are truncations of

                              trnrrrd VSUX

                              VSU

                              )min(1 dnss

                              Review of Linear Algebra (Cont)

                              SVD Full Representation

                              =

                              Graphics Display Assumes

                              ndX ddU ndS nn

                              tV

                              nd

                              Review of Linear Algebra (Cont)

                              SVD Full Representation

                              =

                              Full Rank Basis Matrix

                              ndX ddU ndS nn

                              tV

                              Review of Linear Algebra (Cont)

                              SVD Full Representation

                              =

                              Full Rank Basis Matrix

                              All 0s in Bottom

                              ndX ddU ndS nn

                              tV

                              Review of Linear Algebra (Cont)

                              SVD Reduced Representation

                              =

                              These Columns Get 0ed Out

                              ndX ddU nnS nn

                              tV

                              nnd 0

                              Review of Linear Algebra (Cont)

                              SVD Reduced Representation

                              =ndX ndU nnS nn

                              tV

                              Review of Linear Algebra (Cont)

                              SVD Reduced Representation

                              =

                              Also Some of These May be 0

                              ndX ndU nnS nn

                              tV

                              Review of Linear Algebra (Cont)

                              SVD Compact Representation

                              = ndX rdU

                              rrS nrtV

                              0

                              Review of Linear Algebra (Cont)

                              SVD Compact Representation

                              =

                              These Get 0ed Out

                              ndX rdU

                              rrS nrtV

                              0

                              Review of Linear Algebra (Cont)

                              SVD Compact Representation

                              = ndX rdU

                              rrS nrtV

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition

                              For a (Symmetric) Square Matrix

                              Find

                              ddX

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition

                              For a (Symmetric) Square Matrix

                              Find a Diagonal Matrix

                              Called Eigenvalues

                              Convenient Ordering

                              ddX

                              d

                              D

                              0

                              01

                              n 1

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition

                              For a (Symmetric) Square Matrix

                              Find a Diagonal Matrix

                              And an Orthonormal Matrix

                              (ie )

                              ddX

                              d

                              D

                              0

                              01

                              ddB

                              ddtt IBBBB

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition

                              For a (Symmetric) Square Matrix

                              Find a Diagonal Matrix

                              And an Orthonormal Matrix

                              (ie )

                              So that ie

                              ddX

                              d

                              D

                              0

                              01

                              ddB

                              ddtt IBBBB

                              DBBX tBDBX

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                              (looks similar)

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                              Complex (uses )

                              VU

                              1i

                              Review of Linear Algebra (Cont)

                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                              VU

                              X

                              BVU

                              1i

                              • Participant Presentations
                              • PCA to find clusters
                              • Statistical Smoothing
                              • Kernel Density Estimation
                              • Scatterplot Smoothing
                              • Statistical Smoothing (2)
                              • SiZer Background
                              • SiZer Background (2)
                              • SiZer Background (3)
                              • SiZer Background (4)
                              • SiZer Background (5)
                              • SiZer Overview
                              • PCA to find clusters (2)
                              • PCA to find clusters (3)
                              • PCA to find clusters (4)
                              • PCA to find clusters (5)
                              • PCA to find clusters (6)
                              • PCA to find clusters (7)
                              • Recall Yeast Cell Cycle Data
                              • Yeast Cell Cycle Data FDA View
                              • Yeast Cell Cycle Data FDA View (2)
                              • Yeast Cell Cycle Data FDA View (3)
                              • Yeast Cell Cycles Freq 2 Proj
                              • Frequency 2 Analysis
                              • Frequency 2 Analysis (2)
                              • Frequency 2 Analysis (3)
                              • Yeast Cell Cycle
                              • SiZer Study of Distrsquon of Angles
                              • Reclassification of Major Genes
                              • Compare to Previous Classifrsquon
                              • New Subpopulation View
                              • New Subpopulation View (2)
                              • Detailed Look at PCA
                              • PCA Rediscovery ndash Renaming
                              • PCA Rediscovery ndash Renaming (2)
                              • PCA Rediscovery ndash Renaming (3)
                              • PCA Rediscovery ndash Renaming (4)
                              • PCA Rediscovery ndash Renaming (5)
                              • An Interesting Historical Note
                              • An Interesting Historical Note (2)
                              • Detailed Look at PCA (2)
                              • Detailed Look at PCA (3)
                              • Review of Linear Algebra
                              • Review of Linear Algebra (2)
                              • Review of Linear Algebra (3)
                              • Review of Linear Algebra (Cont)
                              • Review of Linear Algebra (Cont) (2)
                              • Review of Linear Algebra (Cont) (3)
                              • Review of Linear Algebra (Cont) (4)
                              • Review of Linear Algebra (Cont) (5)
                              • Review of Linear Algebra (Cont) (6)
                              • Review of Linear Algebra (Cont) (7)
                              • Review of Linear Algebra (Cont) (8)
                              • Review of Linear Algebra (Cont) (9)
                              • Review of Linear Algebra (Cont) (10)
                              • Review of Linear Algebra (Cont) (11)
                              • Review of Linear Algebra (Cont) (12)
                              • Review of Linear Algebra (Cont) (13)
                              • Review of Linear Algebra (Cont) (14)
                              • Review of Linear Algebra (Cont) (15)
                              • Review of Linear Algebra (Cont) (16)
                              • Review of Linear Algebra (Cont) (17)
                              • Review of Linear Algebra (Cont) (18)
                              • Review of Linear Algebra (Cont) (19)
                              • Review of Linear Algebra (Cont) (20)
                              • Review of Linear Algebra (Cont) (21)
                              • Review of Linear Algebra (Cont) (22)
                              • Review of Linear Algebra (Cont) (23)
                              • Review of Linear Algebra (Cont) (24)
                              • Review of Linear Algebra (Cont) (25)
                              • Review of Linear Algebra (Cont) (26)
                              • Review of Linear Algebra (Cont) (27)
                              • Review of Linear Algebra (Cont) (28)
                              • Review of Linear Algebra (Cont) (29)
                              • Review of Linear Algebra (Cont) (30)
                              • Review of Linear Algebra (Cont) (31)
                              • Review of Linear Algebra (Cont) (32)
                              • Review of Linear Algebra (Cont) (33)
                              • Review of Linear Algebra (Cont) (34)
                              • Review of Linear Algebra (Cont) (35)
                              • Review of Linear Algebra (Cont) (36)
                              • Review of Linear Algebra (Cont) (37)
                              • Review of Linear Algebra (Cont) (38)
                              • Review of Linear Algebra (Cont) (39)
                              • Review of Linear Algebra (Cont) (40)
                              • Review of Linear Algebra (Cont) (41)
                              • Review of Linear Algebra (Cont) (42)
                              • Review of Linear Algebra (Cont) (43)
                              • Review of Linear Algebra (Cont) (44)
                              • Review of Linear Algebra (Cont) (45)
                              • Review of Linear Algebra (Cont) (46)
                              • Review of Linear Algebra (Cont) (47)
                              • Review of Linear Algebra (Cont) (48)
                              • Review of Linear Algebra (Cont) (49)
                              • Review of Linear Algebra (Cont) (50)
                              • Review of Linear Algebra (Cont) (51)
                              • Review of Linear Algebra (Cont) (52)
                              • Review of Linear Algebra (Cont) (53)
                              • Review of Linear Algebra (Cont) (54)
                              • Review of Linear Algebra (Cont) (55)
                              • Review of Linear Algebra (Cont) (56)
                              • Review of Linear Algebra (Cont) (57)
                              • Review of Linear Algebra (Cont) (58)
                              • Review of Linear Algebra (Cont) (59)
                              • Review of Linear Algebra (Cont) (60)
                              • Review of Linear Algebra (Cont) (61)
                              • Review of Linear Algebra (Cont) (62)
                              • Review of Linear Algebra (Cont) (63)
                              • Review of Linear Algebra (Cont) (64)
                              • Review of Linear Algebra (Cont) (65)
                              • Review of Linear Algebra (Cont) (66)
                              • Review of Linear Algebra (Cont) (67)
                              • Review of Linear Algebra (Cont) (68)
                              • Review of Linear Algebra (Cont) (69)
                              • Review of Linear Algebra (Cont) (70)
                              • Review of Linear Algebra (Cont) (71)
                              • Review of Linear Algebra (Cont) (72)
                              • Review of Linear Algebra (Cont) (73)
                              • Review of Linear Algebra (Cont) (74)
                              • Review of Linear Algebra (Cont) (75)
                              • Review of Linear Algebra (Cont) (76)
                              • Review of Linear Algebra (Cont) (77)
                              • Review of Linear Algebra (Cont) (78)
                              • Review of Linear Algebra (Cont) (79)
                              • Review of Linear Algebra (Cont) (80)
                              • Review of Linear Algebra (Cont) (81)

                                PCA to find clustersSiZer analysis of Mass Flux PC1

                                Also in Curvature

                                PCA to find clustersSiZer analysis of Mass Flux PC1

                                And in Other Comprsquos

                                PCA to find clustersSiZer analysis of Mass Flux PC1

                                Conclusion

                                bull Found 3 significant clusters

                                bull Correspond to 3 known ldquocloud typesrdquo

                                bull Worth deeper investigation

                                Recall Yeast Cell Cycle Data

                                bull ldquoGene Expressionrdquo ndash Micro-array data

                                bull Data (after major preprocessing) Expression ldquolevelrdquo of

                                bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                                10s)bull Interesting statistical issue

                                High Dimension Low Sample Size data

                                (HDLSS)

                                Yeast Cell Cycle Data FDA View

                                Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                                Yeast Cell Cycle Data FDA View

                                Periodic genes

                                Naiumlve

                                approach

                                Simple PCA

                                Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                ldquovariationrdquobull PCA finds ldquodirections of maximal

                                variationrdquobull Often but not always same as

                                ldquointeresting directionsrdquobull Here need better approach to study

                                periodicities

                                Yeast Cell Cycles Freq 2 Proj

                                PCA on

                                Freq 2

                                Periodic

                                Component

                                Of Data

                                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                Approach from Zhao Marron amp Wells (2004)

                                Frequency 2 Analysis

                                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                notable differencesbull Now try to improve ldquophase classificationrdquo

                                Yeast Cell CycleRevisit ldquophase classificationrdquo

                                approachbull Use outer 200 genes

                                (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                (finds significant bumps etc in histogram)

                                bull Carefully redrew boundariesbull Check by studying kde angles

                                SiZer Study of Distrsquon of Angles

                                Reclassification of Major Genes

                                Compare to Previous Classifrsquon

                                New Subpopulation View

                                New Subpopulation View

                                NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                Detailed Look at PCA

                                Now Study ldquoFolklorerdquo More Carefully

                                bull BackGround

                                bull History

                                bull Underpinnings

                                (Mathematical amp Computational)

                                Good Overall Reference Jolliffe (2002)

                                PCA Rediscovery ndash Renaming

                                Statistics Principal Component Analysis (PCA)

                                PCA Rediscovery ndash Renaming

                                Statistics Principal Component Analysis (PCA)

                                Social Sciences Factor Analysis (PCA is a subset)

                                PCA Rediscovery ndash Renaming

                                Statistics Principal Component Analysis (PCA)

                                Social Sciences Factor Analysis (PCA is a subset)

                                Probability Electrical EngKarhunen ndash Loeve expansion

                                PCA Rediscovery ndash Renaming

                                Statistics Principal Component Analysis (PCA)

                                Social Sciences Factor Analysis (PCA is a subset)

                                Probability Electrical EngKarhunen ndash Loeve expansion

                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                PCA Rediscovery ndash Renaming

                                Statistics Principal Component Analysis (PCA)

                                Social Sciences Factor Analysis (PCA is a subset)

                                Probability Electrical EngKarhunen ndash Loeve expansion

                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                Geo-Sciences Empirical Orthogonal Functions (EOF)

                                An Interesting Historical Note

                                The 1st () application of PCA to Functional

                                Data Analysis

                                An Interesting Historical Note

                                The 1st () application of PCA to Functional

                                Data Analysis

                                Rao (1958)

                                1st Paper with ldquoCurves as Data Objectsrdquo

                                viewpoint

                                Detailed Look at PCA

                                Three Important (amp Interesting) Viewpoints

                                1 Mathematics

                                2 Numerics

                                3 Statistics

                                Goal Study Interrelationships

                                Detailed Look at PCA

                                Three Important (amp Interesting) Viewpoints

                                1 Mathematics

                                2 Numerics

                                3 Statistics

                                1st Review Linear Alg and Multivar Prob

                                Review of Linear Algebra

                                Vector Space

                                bull set of ldquovectorsrdquo

                                bull and ldquoscalarsrdquo (coefficients)

                                xa

                                Review of Linear Algebra

                                Vector Space

                                bull set of ldquovectorsrdquo

                                bull and ldquoscalarsrdquo (coefficients)

                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                ( in space)

                                xa

                                i

                                ii xa

                                Review of Linear Algebra

                                Vector Space

                                bull set of ldquovectorsrdquo

                                bull and ldquoscalarsrdquo (coefficients)

                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                ( in space)

                                eg

                                ldquo dim Euclidrsquon spacerdquo

                                xa

                                i

                                ii xa

                                d

                                d

                                d xx

                                x

                                x

                                x 1

                                1

                                d

                                Review of Linear Algebra (Cont)

                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                Review of Linear Algebra (Cont)

                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                Review of Linear Algebra (Cont)

                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                Note Planes not Through the Origin

                                are not Subspaces

                                (Do not Contain )00 x

                                Review of Linear Algebra (Cont)

                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                (all Linear Combos of them =

                                = Containing Hyperplane

                                through Origin)

                                Review of Linear Algebra (Cont)

                                Basis of Subspace Set of Vectors that

                                bull Span ie Everything is a Lin Com of them

                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                Review of Linear Algebra (Cont)

                                Basis of Subspace Set of Vectors that

                                bull Span ie Everything is a Lin Com of them

                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                bull eg ldquoUnit Vector Basisrdquo

                                bull Since

                                d

                                1

                                0

                                0

                                0

                                1

                                0

                                0

                                0

                                1

                                1

                                0

                                0

                                0

                                1

                                0

                                0

                                0

                                1

                                212

                                1

                                d

                                d

                                xxx

                                x

                                x

                                x

                                Review of Linear Algebra (Cont)

                                Basis Matrix of subspace of

                                Given a basis

                                create matrix of columns

                                dnvv 1

                                nddnd

                                n

                                n

                                vv

                                vv

                                vvB

                                1

                                111

                                1

                                Review of Linear Algebra (Cont)

                                Then linear combo is a matrix multiplicatrsquon

                                where

                                n

                                iii aBva

                                1

                                na

                                a

                                a 1

                                Review of Linear Algebra (Cont)

                                Then linear combo is a matrix multiplicatrsquon

                                where

                                Note Right Multiplication Gives

                                Linear Combination of Column Vectors

                                n

                                iii aBva

                                1

                                na

                                a

                                a 1

                                Review of Linear Algebra (Cont)

                                Then linear combo is a matrix multiplicatrsquon

                                where

                                Check sizes

                                n

                                iii aBva

                                1

                                na

                                a

                                a 1

                                )1()(1 nndd

                                Review of Linear Algebra (Cont)

                                Aside on Matrix Multiplication (linear transformatrsquon)

                                For matrices

                                Define the Matrix Product

                                mkk

                                m

                                aa

                                aa

                                A

                                1

                                111

                                nmm

                                n

                                bb

                                bb

                                B

                                1

                                111

                                m

                                iniik

                                m

                                iiik

                                m

                                inii

                                m

                                iii

                                baba

                                baba

                                AB

                                1

                                11

                                11

                                111

                                Review of Linear Algebra (Cont)

                                Aside on Matrix Multiplication (linear transformatrsquon)

                                For matrices

                                Define the Matrix Product

                                (Inner Products of Rows With Columns )

                                mkk

                                m

                                aa

                                aa

                                A

                                1

                                111

                                nmm

                                n

                                bb

                                bb

                                B

                                1

                                111

                                m

                                iniik

                                m

                                iiik

                                m

                                inii

                                m

                                iii

                                baba

                                baba

                                AB

                                1

                                11

                                11

                                111

                                A B

                                Review of Linear Algebra (Cont)

                                Aside on Matrix Multiplication (linear transformatrsquon)

                                For matrices

                                Define the Matrix Product

                                (Inner Products of Rows With Columns )

                                (Composition of Linear Transformations)

                                mkk

                                m

                                aa

                                aa

                                A

                                1

                                111

                                nmm

                                n

                                bb

                                bb

                                B

                                1

                                111

                                m

                                iniik

                                m

                                iiik

                                m

                                inii

                                m

                                iii

                                baba

                                baba

                                AB

                                1

                                11

                                11

                                111

                                A B

                                Review of Linear Algebra (Cont)

                                Aside on Matrix Multiplication (linear transformatrsquon)

                                For matrices

                                Define the Matrix Product

                                (Inner Products of Rows With Columns )

                                (Composition of Linear Transformations)

                                Often Useful to Check Sizes

                                mkk

                                m

                                aa

                                aa

                                A

                                1

                                111

                                nmm

                                n

                                bb

                                bb

                                B

                                1

                                111

                                m

                                iniik

                                m

                                iiik

                                m

                                inii

                                m

                                iii

                                baba

                                baba

                                AB

                                1

                                11

                                11

                                111

                                nmmknk

                                A B

                                Review of Linear Algebra (Cont)

                                Aside on Matrix Multiplication (linear transformatrsquon)

                                For matrices

                                Define the Matrix Product

                                (Inner Products of Rows With Columns )

                                (Composition of Linear Transformations)

                                Often Useful to Check Sizes

                                mkk

                                m

                                aa

                                aa

                                A

                                1

                                111

                                nmm

                                n

                                bb

                                bb

                                B

                                1

                                111

                                m

                                iniik

                                m

                                iiik

                                m

                                inii

                                m

                                iii

                                baba

                                baba

                                AB

                                1

                                11

                                11

                                111

                                nmmknk

                                A B

                                Review of Linear Algebra (Cont)

                                Matrix Trace

                                bull For a Square Matrix

                                bull Define

                                mmm

                                m

                                aa

                                aa

                                A

                                1

                                111

                                m

                                iiiaAtr

                                1)(

                                Review of Linear Algebra (Cont)

                                Matrix Trace

                                bull For a Square Matrix

                                bull Define

                                bull Trace Commutes with Matrix Multiplication

                                mmm

                                m

                                aa

                                aa

                                A

                                1

                                111

                                m

                                iiiaAtr

                                1)(

                                BAtrABtr

                                Review of Linear Algebra (Cont)

                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                bull Number of Elements in a Basis (Unique)

                                Review of Linear Algebra (Cont)

                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                bull Number of Elements in a Basis (Unique)

                                bull (Use Basis Above)

                                bull eg dim of a line is 1

                                bull eg dim of a plane is 2

                                dd dim

                                Review of Linear Algebra (Cont)

                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                bull Number of Elements in a Basis (Unique)

                                bull (Use Basis Above)

                                bull eg dim of a line is 1

                                bull eg dim of a plane is 2

                                bull Dimension is ldquoDegrees of Freedomrdquo

                                (in Statistical Uses eg ANOVA)

                                dd dim

                                Review of Linear Algebra (Cont)

                                Norm of a Vector

                                bull in d 21

                                21

                                1

                                2 xxxx td

                                jj

                                Review of Linear Algebra (Cont)

                                Norm of a Vector

                                bull in

                                bull Idea length of the vector

                                d 2121

                                1

                                2 xxxx td

                                jj

                                Review of Linear Algebra (Cont)

                                Norm of a Vector

                                bull in

                                bull Idea length of the vector

                                bull Note strange properties for high

                                eg ldquolength of diagonal of unit cuberdquo =

                                d 2121

                                1

                                2 xxxx td

                                jj

                                d

                                d

                                Review of Linear Algebra (Cont)

                                Norm of a Vector (cont)

                                bull Length Normalized Vector

                                (has Length 1 thus on Surf of Unit Sphere

                                amp is a Direction Vector)

                                x

                                x

                                Review of Linear Algebra (Cont)

                                Norm of a Vector (cont)

                                bull Length Normalized Vector

                                (has Length 1 thus on Surf of Unit Sphere

                                amp is a Direction Vector)

                                bull Define Distance as

                                x

                                x

                                yxyxyxyxd t

                                Review of Linear Algebra (Cont)

                                Inner (Dot Scalar) Product

                                bull for Vectors and

                                yxyxyx td

                                jjj

                                1

                                x y

                                Review of Linear Algebra (Cont)

                                Inner (Dot Scalar) Product

                                bull for Vectors and

                                bull Related to Norm via

                                yxyxyx td

                                jjj

                                1

                                x y

                                21 xxx

                                Review of Linear Algebra (Cont)

                                Inner (Dot Scalar) Product (cont)

                                bull measures ldquoangle between and rdquo as

                                yyxx

                                yx

                                yx

                                yxyxangle

                                tt

                                t

                                11 cos

                                cos

                                x y

                                Review of Linear Algebra (Cont)

                                Inner (Dot Scalar) Product (cont)

                                bull measures ldquoangle between and rdquo as

                                bull key to Orthogonality ie Perpendiculrsquoty

                                if and only if

                                yyxx

                                yx

                                yx

                                yxyxangle

                                tt

                                t

                                11 cos

                                cos

                                x y

                                yx 0 yx

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis

                                bull All Orthogonal to each other

                                ie for

                                bull All have Length 1

                                ie for

                                nvv 1

                                1 ii vv

                                0 ii vv ii

                                ni 1

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis (cont)

                                bull Spectral Representation

                                where

                                nvv 1

                                n

                                iii vax

                                1

                                ii vxa

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis (cont)

                                bull Spectral Representation

                                where

                                (Coefficient is Inner Product Cool Notation)

                                nvv 1

                                n

                                iii vax

                                1

                                ii vxa

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis (cont)

                                bull Spectral Representation

                                where

                                Check

                                nvv 1

                                n

                                iii vax

                                1

                                ii vxa

                                iii

                                n

                                iii

                                n

                                iiii avvavvavx

                                1

                                1

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis (cont)

                                bull Spectral Representation

                                where

                                Check

                                bull Matrix Notation where ie

                                For the Basis Matrix

                                nvv 1

                                n

                                iii vax

                                1

                                ii vxa

                                iii

                                n

                                iii

                                n

                                iiii avvavvavx

                                1

                                1

                                aBx Bxa tt xBa t

                                nvvB 1

                                Review of Linear Algebra (Cont)

                                Orthonormal Basis (cont)

                                bull Spectral Representation

                                where

                                Check

                                bull Matrix Notation where ie

                                is called transform of

                                (eg Fourier or Wavelet)

                                nvv 1

                                n

                                iii vax

                                1

                                ii vxa

                                iii

                                n

                                iii

                                n

                                iiii avvavvavx

                                1

                                1

                                aBx Bxa tt xBa t

                                xa

                                Review of Linear Algebra (Cont)

                                Parseval identity for

                                in subsp genrsquod by o n basis

                                x

                                nvv 1

                                2

                                1

                                22

                                1

                                2 aavxx

                                n

                                ii

                                n

                                ii

                                Review of Linear Algebra (Cont)

                                Parseval identity for

                                in subsp genrsquod by o n basis

                                bull Pythagorean theorem

                                bull ldquoDecomposition of Energyrdquo

                                bull ANOVA - sums of squares

                                x

                                nvv 1

                                2

                                1

                                22

                                1

                                2 aavxx

                                n

                                ii

                                n

                                ii

                                Review of Linear Algebra (Cont)

                                Parseval identity for

                                in subsp genrsquod by o n basis

                                bull Pythagorean theorem

                                bull ldquoDecomposition of Energyrdquo

                                bull ANOVA - sums of squares

                                bull Transform has same length as

                                ie ldquorotation in rdquo

                                x

                                nvv 1

                                2

                                1

                                22

                                1

                                2 aavxx

                                n

                                ii

                                n

                                ii

                                a xd

                                Projection of a Vector onto a Subspace

                                bull Idea Member of that is Closest to

                                (ie ldquoBest Approxrsquonrdquo)

                                Review of Linear Algebra (Cont)x

                                xV

                                V

                                Projection of a Vector onto a Subspace

                                bull Idea Member of that is Closest to

                                (ie ldquoBest Approxrsquonrdquo)

                                bull Find that Solves

                                (ldquoLeast Squaresrdquo)

                                Review of Linear Algebra (Cont)x

                                xV

                                V

                                VxPV vxVv

                                min

                                Projection of a Vector onto a Subspace

                                bull Idea Member of that is Closest to

                                (ie ldquoBest Approxrsquonrdquo)

                                bull Find that Solves

                                (ldquoLeast Squaresrdquo)

                                bull For Inner Product (Hilbert) Space

                                Exists and is Unique

                                Review of Linear Algebra (Cont)x

                                xV

                                V

                                VxPV vxVv

                                min

                                xPV

                                Projection of a Vector onto a Subspace (cont)

                                bull General Solution in for Basis Matrix

                                Review of Linear Algebra (Cont)

                                d VB

                                xBBBBxP tVV

                                tVVV

                                1

                                Projection of a Vector onto a Subspace (cont)

                                bull General Solution in for Basis Matrix

                                bull So Projrsquon Operator is Matrix Multrsquon

                                (thus projection is another linear operation)

                                Review of Linear Algebra (Cont)

                                d VB

                                xBBBBxP tVV

                                tVVV

                                1

                                tVV

                                tVVV BBBBP

                                1

                                Projection of a Vector onto a Subspace (cont)

                                bull General Solution in for Basis Matrix

                                bull So Projrsquon Operator is Matrix Multrsquon

                                (thus projection is another linear operation)

                                (note same operation underlies least squares)

                                Review of Linear Algebra (Cont)

                                d VB

                                xBBBBxP tVV

                                tVVV

                                1

                                tVV

                                tVVV BBBBP

                                1

                                Review of Linear Algebra (Cont)

                                Projection using Orthonormal Basis

                                bull Basis Matrix is Orthonormal

                                nnVtV IBB

                                10

                                01

                                1

                                111

                                1

                                1

                                nnn

                                n

                                ntn

                                t

                                vvvv

                                vvvv

                                vv

                                v

                                v

                                nvv 1

                                Review of Linear Algebra (Cont)

                                Projection using Orthonormal Basis

                                bull Basis Matrix is Orthonormal

                                bull So =

                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                (Recall Right Multrsquon)

                                nnVtV IBB

                                10

                                01

                                1

                                111

                                1

                                1

                                nnn

                                n

                                ntn

                                t

                                vvvv

                                vvvv

                                vv

                                v

                                v

                                xBBxP tVVV

                                x V

                                nvv 1

                                Review of Linear Algebra (Cont)

                                Projection using Orthonormal Basis (cont)

                                bull For Orthogonal Complement

                                and

                                V

                                xPxPx VV 222xPxPx VV

                                Review of Linear Algebra (Cont)

                                Projection using Orthonormal Basis (cont)

                                bull For Orthogonal Complement

                                and

                                bull Parseval Inequality

                                V

                                xPxPx VV 222xPxPx VV

                                2

                                1

                                22

                                1

                                22 aavxxxP

                                n

                                ii

                                n

                                iiV

                                Review of Linear Algebra (Cont)

                                (Real) Unitary Matrices with

                                bull Orthonormal Basis Matrix

                                (So All of Above Applies)

                                ddU IUU t

                                Review of Linear Algebra (Cont)

                                (Real) Unitary Matrices with

                                bull Orthonormal Basis Matrix

                                (So All of Above Applies)

                                bull Note Transformrsquon is Distance Preserving

                                ddU IUU t

                                yxdyxyxyxUyUxUdn

                                i ii 2

                                1

                                Review of Linear Algebra (Cont)

                                (Real) Unitary Matrices with

                                bull Orthonormal Basis Matrix

                                (So All of Above Applies)

                                bull Note Transformrsquon is Distance Preserving

                                bull Lin Trans (Mult by ) is ~ Rotation

                                bull But also Includes ldquoMirror Imagesrdquo

                                ddU IUU t

                                yxdyxyxyxUyUxUdn

                                i ii 2

                                1

                                U

                                Review of Linear Algebra (Cont)

                                Singular Value Decomposition (SVD)

                                For a Matrix

                                Find

                                ndX

                                Review of Linear Algebra (Cont)

                                Singular Value Decomposition (SVD)

                                For a Matrix

                                Find a Diagonal Matrix

                                with Entries

                                called Singular Values

                                ndX

                                ndS

                                )min(1 ndss

                                Review of Linear Algebra (Cont)

                                Singular Value Decomposition (SVD)

                                For a Matrix

                                Find a Diagonal Matrix

                                with Entries

                                called Singular Values

                                And Unitary (Rotation) Matrices

                                (recall )

                                ndX

                                ndS

                                )min(1 ndss

                                ddU nnV

                                IVVUU tt

                                Review of Linear Algebra (Cont)

                                Singular Value Decomposition (SVD)

                                For a Matrix

                                Find a Diagonal Matrix

                                with Entries

                                called Singular Values

                                And Unitary (Rotation) Matrices

                                (recall )

                                So That

                                ndX

                                ndS

                                )min(1 ndss

                                ddU nnV

                                IVVUU tt tUSVX

                                Review of Linear Algebra (Cont)

                                Intuition behind Singular Value Decomposition

                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                vVSUvVSUvX tt

                                Review of Linear Algebra (Cont)

                                Intuition behind Singular Value Decomposition

                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                bull First rotate

                                X

                                vVSUvVSUvX tt

                                Review of Linear Algebra (Cont)

                                Intuition behind Singular Value Decomposition

                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                bull First rotate

                                bull Second rescale coordinate axes (by )

                                X

                                vVSUvVSUvX tt

                                is

                                Review of Linear Algebra (Cont)

                                Intuition behind Singular Value Decomposition

                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                bull First rotate

                                bull Second rescale coordinate axes (by )

                                bull Third rotate again

                                X

                                vVSUvVSUvX tt

                                is

                                Review of Linear Algebra (Cont)

                                Intuition behind Singular Value Decomposition

                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                bull First rotate

                                bull Second rescale coordinate axes (by )

                                bull Third rotate again

                                bull ie have diagonalized the transformation

                                X

                                vVSUvVSUvX tt

                                is

                                SVD Compact Representation

                                Useful Labeling

                                Singular Values in Increasing Order

                                Review of Linear Algebra (Cont)

                                )min(1 dnss

                                SVD Compact Representation

                                Useful Labeling

                                Singular Values in Increasing Order

                                Note singular values = 0 can be omitted

                                (Since do ldquo0-Stretchingrdquo)

                                Review of Linear Algebra (Cont)

                                )min(1 dnss

                                Review of Linear Algebra (Cont)

                                r

                                SVD Compact Representation

                                Useful Labeling

                                Singular Values in Increasing Order

                                Note singular values = 0 can be omitted

                                Let = of positive singular values

                                )min(1 dnss

                                Review of Linear Algebra (Cont)

                                r

                                SVD Compact Representation

                                Useful Labeling

                                Singular Values in Increasing Order

                                Note singular values = 0 can be omitted

                                Let = of positive singular values

                                Then

                                Where are truncations of

                                trnrrrd VSUX

                                VSU

                                )min(1 dnss

                                Review of Linear Algebra (Cont)

                                SVD Full Representation

                                =

                                Graphics Display Assumes

                                ndX ddU ndS nn

                                tV

                                nd

                                Review of Linear Algebra (Cont)

                                SVD Full Representation

                                =

                                Full Rank Basis Matrix

                                ndX ddU ndS nn

                                tV

                                Review of Linear Algebra (Cont)

                                SVD Full Representation

                                =

                                Full Rank Basis Matrix

                                All 0s in Bottom

                                ndX ddU ndS nn

                                tV

                                Review of Linear Algebra (Cont)

                                SVD Reduced Representation

                                =

                                These Columns Get 0ed Out

                                ndX ddU nnS nn

                                tV

                                nnd 0

                                Review of Linear Algebra (Cont)

                                SVD Reduced Representation

                                =ndX ndU nnS nn

                                tV

                                Review of Linear Algebra (Cont)

                                SVD Reduced Representation

                                =

                                Also Some of These May be 0

                                ndX ndU nnS nn

                                tV

                                Review of Linear Algebra (Cont)

                                SVD Compact Representation

                                = ndX rdU

                                rrS nrtV

                                0

                                Review of Linear Algebra (Cont)

                                SVD Compact Representation

                                =

                                These Get 0ed Out

                                ndX rdU

                                rrS nrtV

                                0

                                Review of Linear Algebra (Cont)

                                SVD Compact Representation

                                = ndX rdU

                                rrS nrtV

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition

                                For a (Symmetric) Square Matrix

                                Find

                                ddX

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition

                                For a (Symmetric) Square Matrix

                                Find a Diagonal Matrix

                                Called Eigenvalues

                                Convenient Ordering

                                ddX

                                d

                                D

                                0

                                01

                                n 1

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition

                                For a (Symmetric) Square Matrix

                                Find a Diagonal Matrix

                                And an Orthonormal Matrix

                                (ie )

                                ddX

                                d

                                D

                                0

                                01

                                ddB

                                ddtt IBBBB

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition

                                For a (Symmetric) Square Matrix

                                Find a Diagonal Matrix

                                And an Orthonormal Matrix

                                (ie )

                                So that ie

                                ddX

                                d

                                D

                                0

                                01

                                ddB

                                ddtt IBBBB

                                DBBX tBDBX

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                (looks similar)

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                Complex (uses )

                                VU

                                1i

                                Review of Linear Algebra (Cont)

                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                VU

                                X

                                BVU

                                1i

                                • Participant Presentations
                                • PCA to find clusters
                                • Statistical Smoothing
                                • Kernel Density Estimation
                                • Scatterplot Smoothing
                                • Statistical Smoothing (2)
                                • SiZer Background
                                • SiZer Background (2)
                                • SiZer Background (3)
                                • SiZer Background (4)
                                • SiZer Background (5)
                                • SiZer Overview
                                • PCA to find clusters (2)
                                • PCA to find clusters (3)
                                • PCA to find clusters (4)
                                • PCA to find clusters (5)
                                • PCA to find clusters (6)
                                • PCA to find clusters (7)
                                • Recall Yeast Cell Cycle Data
                                • Yeast Cell Cycle Data FDA View
                                • Yeast Cell Cycle Data FDA View (2)
                                • Yeast Cell Cycle Data FDA View (3)
                                • Yeast Cell Cycles Freq 2 Proj
                                • Frequency 2 Analysis
                                • Frequency 2 Analysis (2)
                                • Frequency 2 Analysis (3)
                                • Yeast Cell Cycle
                                • SiZer Study of Distrsquon of Angles
                                • Reclassification of Major Genes
                                • Compare to Previous Classifrsquon
                                • New Subpopulation View
                                • New Subpopulation View (2)
                                • Detailed Look at PCA
                                • PCA Rediscovery ndash Renaming
                                • PCA Rediscovery ndash Renaming (2)
                                • PCA Rediscovery ndash Renaming (3)
                                • PCA Rediscovery ndash Renaming (4)
                                • PCA Rediscovery ndash Renaming (5)
                                • An Interesting Historical Note
                                • An Interesting Historical Note (2)
                                • Detailed Look at PCA (2)
                                • Detailed Look at PCA (3)
                                • Review of Linear Algebra
                                • Review of Linear Algebra (2)
                                • Review of Linear Algebra (3)
                                • Review of Linear Algebra (Cont)
                                • Review of Linear Algebra (Cont) (2)
                                • Review of Linear Algebra (Cont) (3)
                                • Review of Linear Algebra (Cont) (4)
                                • Review of Linear Algebra (Cont) (5)
                                • Review of Linear Algebra (Cont) (6)
                                • Review of Linear Algebra (Cont) (7)
                                • Review of Linear Algebra (Cont) (8)
                                • Review of Linear Algebra (Cont) (9)
                                • Review of Linear Algebra (Cont) (10)
                                • Review of Linear Algebra (Cont) (11)
                                • Review of Linear Algebra (Cont) (12)
                                • Review of Linear Algebra (Cont) (13)
                                • Review of Linear Algebra (Cont) (14)
                                • Review of Linear Algebra (Cont) (15)
                                • Review of Linear Algebra (Cont) (16)
                                • Review of Linear Algebra (Cont) (17)
                                • Review of Linear Algebra (Cont) (18)
                                • Review of Linear Algebra (Cont) (19)
                                • Review of Linear Algebra (Cont) (20)
                                • Review of Linear Algebra (Cont) (21)
                                • Review of Linear Algebra (Cont) (22)
                                • Review of Linear Algebra (Cont) (23)
                                • Review of Linear Algebra (Cont) (24)
                                • Review of Linear Algebra (Cont) (25)
                                • Review of Linear Algebra (Cont) (26)
                                • Review of Linear Algebra (Cont) (27)
                                • Review of Linear Algebra (Cont) (28)
                                • Review of Linear Algebra (Cont) (29)
                                • Review of Linear Algebra (Cont) (30)
                                • Review of Linear Algebra (Cont) (31)
                                • Review of Linear Algebra (Cont) (32)
                                • Review of Linear Algebra (Cont) (33)
                                • Review of Linear Algebra (Cont) (34)
                                • Review of Linear Algebra (Cont) (35)
                                • Review of Linear Algebra (Cont) (36)
                                • Review of Linear Algebra (Cont) (37)
                                • Review of Linear Algebra (Cont) (38)
                                • Review of Linear Algebra (Cont) (39)
                                • Review of Linear Algebra (Cont) (40)
                                • Review of Linear Algebra (Cont) (41)
                                • Review of Linear Algebra (Cont) (42)
                                • Review of Linear Algebra (Cont) (43)
                                • Review of Linear Algebra (Cont) (44)
                                • Review of Linear Algebra (Cont) (45)
                                • Review of Linear Algebra (Cont) (46)
                                • Review of Linear Algebra (Cont) (47)
                                • Review of Linear Algebra (Cont) (48)
                                • Review of Linear Algebra (Cont) (49)
                                • Review of Linear Algebra (Cont) (50)
                                • Review of Linear Algebra (Cont) (51)
                                • Review of Linear Algebra (Cont) (52)
                                • Review of Linear Algebra (Cont) (53)
                                • Review of Linear Algebra (Cont) (54)
                                • Review of Linear Algebra (Cont) (55)
                                • Review of Linear Algebra (Cont) (56)
                                • Review of Linear Algebra (Cont) (57)
                                • Review of Linear Algebra (Cont) (58)
                                • Review of Linear Algebra (Cont) (59)
                                • Review of Linear Algebra (Cont) (60)
                                • Review of Linear Algebra (Cont) (61)
                                • Review of Linear Algebra (Cont) (62)
                                • Review of Linear Algebra (Cont) (63)
                                • Review of Linear Algebra (Cont) (64)
                                • Review of Linear Algebra (Cont) (65)
                                • Review of Linear Algebra (Cont) (66)
                                • Review of Linear Algebra (Cont) (67)
                                • Review of Linear Algebra (Cont) (68)
                                • Review of Linear Algebra (Cont) (69)
                                • Review of Linear Algebra (Cont) (70)
                                • Review of Linear Algebra (Cont) (71)
                                • Review of Linear Algebra (Cont) (72)
                                • Review of Linear Algebra (Cont) (73)
                                • Review of Linear Algebra (Cont) (74)
                                • Review of Linear Algebra (Cont) (75)
                                • Review of Linear Algebra (Cont) (76)
                                • Review of Linear Algebra (Cont) (77)
                                • Review of Linear Algebra (Cont) (78)
                                • Review of Linear Algebra (Cont) (79)
                                • Review of Linear Algebra (Cont) (80)
                                • Review of Linear Algebra (Cont) (81)

                                  PCA to find clustersSiZer analysis of Mass Flux PC1

                                  And in Other Comprsquos

                                  PCA to find clustersSiZer analysis of Mass Flux PC1

                                  Conclusion

                                  bull Found 3 significant clusters

                                  bull Correspond to 3 known ldquocloud typesrdquo

                                  bull Worth deeper investigation

                                  Recall Yeast Cell Cycle Data

                                  bull ldquoGene Expressionrdquo ndash Micro-array data

                                  bull Data (after major preprocessing) Expression ldquolevelrdquo of

                                  bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                                  10s)bull Interesting statistical issue

                                  High Dimension Low Sample Size data

                                  (HDLSS)

                                  Yeast Cell Cycle Data FDA View

                                  Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                                  Yeast Cell Cycle Data FDA View

                                  Periodic genes

                                  Naiumlve

                                  approach

                                  Simple PCA

                                  Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                  ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                  ldquovariationrdquobull PCA finds ldquodirections of maximal

                                  variationrdquobull Often but not always same as

                                  ldquointeresting directionsrdquobull Here need better approach to study

                                  periodicities

                                  Yeast Cell Cycles Freq 2 Proj

                                  PCA on

                                  Freq 2

                                  Periodic

                                  Component

                                  Of Data

                                  Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                  cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                  Approach from Zhao Marron amp Wells (2004)

                                  Frequency 2 Analysis

                                  Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                  cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                  classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                  notable differencesbull Now try to improve ldquophase classificationrdquo

                                  Yeast Cell CycleRevisit ldquophase classificationrdquo

                                  approachbull Use outer 200 genes

                                  (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                  (finds significant bumps etc in histogram)

                                  bull Carefully redrew boundariesbull Check by studying kde angles

                                  SiZer Study of Distrsquon of Angles

                                  Reclassification of Major Genes

                                  Compare to Previous Classifrsquon

                                  New Subpopulation View

                                  New Subpopulation View

                                  NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                  Detailed Look at PCA

                                  Now Study ldquoFolklorerdquo More Carefully

                                  bull BackGround

                                  bull History

                                  bull Underpinnings

                                  (Mathematical amp Computational)

                                  Good Overall Reference Jolliffe (2002)

                                  PCA Rediscovery ndash Renaming

                                  Statistics Principal Component Analysis (PCA)

                                  PCA Rediscovery ndash Renaming

                                  Statistics Principal Component Analysis (PCA)

                                  Social Sciences Factor Analysis (PCA is a subset)

                                  PCA Rediscovery ndash Renaming

                                  Statistics Principal Component Analysis (PCA)

                                  Social Sciences Factor Analysis (PCA is a subset)

                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                  PCA Rediscovery ndash Renaming

                                  Statistics Principal Component Analysis (PCA)

                                  Social Sciences Factor Analysis (PCA is a subset)

                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                  PCA Rediscovery ndash Renaming

                                  Statistics Principal Component Analysis (PCA)

                                  Social Sciences Factor Analysis (PCA is a subset)

                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                  Geo-Sciences Empirical Orthogonal Functions (EOF)

                                  An Interesting Historical Note

                                  The 1st () application of PCA to Functional

                                  Data Analysis

                                  An Interesting Historical Note

                                  The 1st () application of PCA to Functional

                                  Data Analysis

                                  Rao (1958)

                                  1st Paper with ldquoCurves as Data Objectsrdquo

                                  viewpoint

                                  Detailed Look at PCA

                                  Three Important (amp Interesting) Viewpoints

                                  1 Mathematics

                                  2 Numerics

                                  3 Statistics

                                  Goal Study Interrelationships

                                  Detailed Look at PCA

                                  Three Important (amp Interesting) Viewpoints

                                  1 Mathematics

                                  2 Numerics

                                  3 Statistics

                                  1st Review Linear Alg and Multivar Prob

                                  Review of Linear Algebra

                                  Vector Space

                                  bull set of ldquovectorsrdquo

                                  bull and ldquoscalarsrdquo (coefficients)

                                  xa

                                  Review of Linear Algebra

                                  Vector Space

                                  bull set of ldquovectorsrdquo

                                  bull and ldquoscalarsrdquo (coefficients)

                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                  ( in space)

                                  xa

                                  i

                                  ii xa

                                  Review of Linear Algebra

                                  Vector Space

                                  bull set of ldquovectorsrdquo

                                  bull and ldquoscalarsrdquo (coefficients)

                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                  ( in space)

                                  eg

                                  ldquo dim Euclidrsquon spacerdquo

                                  xa

                                  i

                                  ii xa

                                  d

                                  d

                                  d xx

                                  x

                                  x

                                  x 1

                                  1

                                  d

                                  Review of Linear Algebra (Cont)

                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                  Review of Linear Algebra (Cont)

                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                  Review of Linear Algebra (Cont)

                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                  Note Planes not Through the Origin

                                  are not Subspaces

                                  (Do not Contain )00 x

                                  Review of Linear Algebra (Cont)

                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                  (all Linear Combos of them =

                                  = Containing Hyperplane

                                  through Origin)

                                  Review of Linear Algebra (Cont)

                                  Basis of Subspace Set of Vectors that

                                  bull Span ie Everything is a Lin Com of them

                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                  Review of Linear Algebra (Cont)

                                  Basis of Subspace Set of Vectors that

                                  bull Span ie Everything is a Lin Com of them

                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                  bull eg ldquoUnit Vector Basisrdquo

                                  bull Since

                                  d

                                  1

                                  0

                                  0

                                  0

                                  1

                                  0

                                  0

                                  0

                                  1

                                  1

                                  0

                                  0

                                  0

                                  1

                                  0

                                  0

                                  0

                                  1

                                  212

                                  1

                                  d

                                  d

                                  xxx

                                  x

                                  x

                                  x

                                  Review of Linear Algebra (Cont)

                                  Basis Matrix of subspace of

                                  Given a basis

                                  create matrix of columns

                                  dnvv 1

                                  nddnd

                                  n

                                  n

                                  vv

                                  vv

                                  vvB

                                  1

                                  111

                                  1

                                  Review of Linear Algebra (Cont)

                                  Then linear combo is a matrix multiplicatrsquon

                                  where

                                  n

                                  iii aBva

                                  1

                                  na

                                  a

                                  a 1

                                  Review of Linear Algebra (Cont)

                                  Then linear combo is a matrix multiplicatrsquon

                                  where

                                  Note Right Multiplication Gives

                                  Linear Combination of Column Vectors

                                  n

                                  iii aBva

                                  1

                                  na

                                  a

                                  a 1

                                  Review of Linear Algebra (Cont)

                                  Then linear combo is a matrix multiplicatrsquon

                                  where

                                  Check sizes

                                  n

                                  iii aBva

                                  1

                                  na

                                  a

                                  a 1

                                  )1()(1 nndd

                                  Review of Linear Algebra (Cont)

                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                  For matrices

                                  Define the Matrix Product

                                  mkk

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  nmm

                                  n

                                  bb

                                  bb

                                  B

                                  1

                                  111

                                  m

                                  iniik

                                  m

                                  iiik

                                  m

                                  inii

                                  m

                                  iii

                                  baba

                                  baba

                                  AB

                                  1

                                  11

                                  11

                                  111

                                  Review of Linear Algebra (Cont)

                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                  For matrices

                                  Define the Matrix Product

                                  (Inner Products of Rows With Columns )

                                  mkk

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  nmm

                                  n

                                  bb

                                  bb

                                  B

                                  1

                                  111

                                  m

                                  iniik

                                  m

                                  iiik

                                  m

                                  inii

                                  m

                                  iii

                                  baba

                                  baba

                                  AB

                                  1

                                  11

                                  11

                                  111

                                  A B

                                  Review of Linear Algebra (Cont)

                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                  For matrices

                                  Define the Matrix Product

                                  (Inner Products of Rows With Columns )

                                  (Composition of Linear Transformations)

                                  mkk

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  nmm

                                  n

                                  bb

                                  bb

                                  B

                                  1

                                  111

                                  m

                                  iniik

                                  m

                                  iiik

                                  m

                                  inii

                                  m

                                  iii

                                  baba

                                  baba

                                  AB

                                  1

                                  11

                                  11

                                  111

                                  A B

                                  Review of Linear Algebra (Cont)

                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                  For matrices

                                  Define the Matrix Product

                                  (Inner Products of Rows With Columns )

                                  (Composition of Linear Transformations)

                                  Often Useful to Check Sizes

                                  mkk

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  nmm

                                  n

                                  bb

                                  bb

                                  B

                                  1

                                  111

                                  m

                                  iniik

                                  m

                                  iiik

                                  m

                                  inii

                                  m

                                  iii

                                  baba

                                  baba

                                  AB

                                  1

                                  11

                                  11

                                  111

                                  nmmknk

                                  A B

                                  Review of Linear Algebra (Cont)

                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                  For matrices

                                  Define the Matrix Product

                                  (Inner Products of Rows With Columns )

                                  (Composition of Linear Transformations)

                                  Often Useful to Check Sizes

                                  mkk

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  nmm

                                  n

                                  bb

                                  bb

                                  B

                                  1

                                  111

                                  m

                                  iniik

                                  m

                                  iiik

                                  m

                                  inii

                                  m

                                  iii

                                  baba

                                  baba

                                  AB

                                  1

                                  11

                                  11

                                  111

                                  nmmknk

                                  A B

                                  Review of Linear Algebra (Cont)

                                  Matrix Trace

                                  bull For a Square Matrix

                                  bull Define

                                  mmm

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  m

                                  iiiaAtr

                                  1)(

                                  Review of Linear Algebra (Cont)

                                  Matrix Trace

                                  bull For a Square Matrix

                                  bull Define

                                  bull Trace Commutes with Matrix Multiplication

                                  mmm

                                  m

                                  aa

                                  aa

                                  A

                                  1

                                  111

                                  m

                                  iiiaAtr

                                  1)(

                                  BAtrABtr

                                  Review of Linear Algebra (Cont)

                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                  bull Number of Elements in a Basis (Unique)

                                  Review of Linear Algebra (Cont)

                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                  bull Number of Elements in a Basis (Unique)

                                  bull (Use Basis Above)

                                  bull eg dim of a line is 1

                                  bull eg dim of a plane is 2

                                  dd dim

                                  Review of Linear Algebra (Cont)

                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                  bull Number of Elements in a Basis (Unique)

                                  bull (Use Basis Above)

                                  bull eg dim of a line is 1

                                  bull eg dim of a plane is 2

                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                  (in Statistical Uses eg ANOVA)

                                  dd dim

                                  Review of Linear Algebra (Cont)

                                  Norm of a Vector

                                  bull in d 21

                                  21

                                  1

                                  2 xxxx td

                                  jj

                                  Review of Linear Algebra (Cont)

                                  Norm of a Vector

                                  bull in

                                  bull Idea length of the vector

                                  d 2121

                                  1

                                  2 xxxx td

                                  jj

                                  Review of Linear Algebra (Cont)

                                  Norm of a Vector

                                  bull in

                                  bull Idea length of the vector

                                  bull Note strange properties for high

                                  eg ldquolength of diagonal of unit cuberdquo =

                                  d 2121

                                  1

                                  2 xxxx td

                                  jj

                                  d

                                  d

                                  Review of Linear Algebra (Cont)

                                  Norm of a Vector (cont)

                                  bull Length Normalized Vector

                                  (has Length 1 thus on Surf of Unit Sphere

                                  amp is a Direction Vector)

                                  x

                                  x

                                  Review of Linear Algebra (Cont)

                                  Norm of a Vector (cont)

                                  bull Length Normalized Vector

                                  (has Length 1 thus on Surf of Unit Sphere

                                  amp is a Direction Vector)

                                  bull Define Distance as

                                  x

                                  x

                                  yxyxyxyxd t

                                  Review of Linear Algebra (Cont)

                                  Inner (Dot Scalar) Product

                                  bull for Vectors and

                                  yxyxyx td

                                  jjj

                                  1

                                  x y

                                  Review of Linear Algebra (Cont)

                                  Inner (Dot Scalar) Product

                                  bull for Vectors and

                                  bull Related to Norm via

                                  yxyxyx td

                                  jjj

                                  1

                                  x y

                                  21 xxx

                                  Review of Linear Algebra (Cont)

                                  Inner (Dot Scalar) Product (cont)

                                  bull measures ldquoangle between and rdquo as

                                  yyxx

                                  yx

                                  yx

                                  yxyxangle

                                  tt

                                  t

                                  11 cos

                                  cos

                                  x y

                                  Review of Linear Algebra (Cont)

                                  Inner (Dot Scalar) Product (cont)

                                  bull measures ldquoangle between and rdquo as

                                  bull key to Orthogonality ie Perpendiculrsquoty

                                  if and only if

                                  yyxx

                                  yx

                                  yx

                                  yxyxangle

                                  tt

                                  t

                                  11 cos

                                  cos

                                  x y

                                  yx 0 yx

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis

                                  bull All Orthogonal to each other

                                  ie for

                                  bull All have Length 1

                                  ie for

                                  nvv 1

                                  1 ii vv

                                  0 ii vv ii

                                  ni 1

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis (cont)

                                  bull Spectral Representation

                                  where

                                  nvv 1

                                  n

                                  iii vax

                                  1

                                  ii vxa

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis (cont)

                                  bull Spectral Representation

                                  where

                                  (Coefficient is Inner Product Cool Notation)

                                  nvv 1

                                  n

                                  iii vax

                                  1

                                  ii vxa

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis (cont)

                                  bull Spectral Representation

                                  where

                                  Check

                                  nvv 1

                                  n

                                  iii vax

                                  1

                                  ii vxa

                                  iii

                                  n

                                  iii

                                  n

                                  iiii avvavvavx

                                  1

                                  1

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis (cont)

                                  bull Spectral Representation

                                  where

                                  Check

                                  bull Matrix Notation where ie

                                  For the Basis Matrix

                                  nvv 1

                                  n

                                  iii vax

                                  1

                                  ii vxa

                                  iii

                                  n

                                  iii

                                  n

                                  iiii avvavvavx

                                  1

                                  1

                                  aBx Bxa tt xBa t

                                  nvvB 1

                                  Review of Linear Algebra (Cont)

                                  Orthonormal Basis (cont)

                                  bull Spectral Representation

                                  where

                                  Check

                                  bull Matrix Notation where ie

                                  is called transform of

                                  (eg Fourier or Wavelet)

                                  nvv 1

                                  n

                                  iii vax

                                  1

                                  ii vxa

                                  iii

                                  n

                                  iii

                                  n

                                  iiii avvavvavx

                                  1

                                  1

                                  aBx Bxa tt xBa t

                                  xa

                                  Review of Linear Algebra (Cont)

                                  Parseval identity for

                                  in subsp genrsquod by o n basis

                                  x

                                  nvv 1

                                  2

                                  1

                                  22

                                  1

                                  2 aavxx

                                  n

                                  ii

                                  n

                                  ii

                                  Review of Linear Algebra (Cont)

                                  Parseval identity for

                                  in subsp genrsquod by o n basis

                                  bull Pythagorean theorem

                                  bull ldquoDecomposition of Energyrdquo

                                  bull ANOVA - sums of squares

                                  x

                                  nvv 1

                                  2

                                  1

                                  22

                                  1

                                  2 aavxx

                                  n

                                  ii

                                  n

                                  ii

                                  Review of Linear Algebra (Cont)

                                  Parseval identity for

                                  in subsp genrsquod by o n basis

                                  bull Pythagorean theorem

                                  bull ldquoDecomposition of Energyrdquo

                                  bull ANOVA - sums of squares

                                  bull Transform has same length as

                                  ie ldquorotation in rdquo

                                  x

                                  nvv 1

                                  2

                                  1

                                  22

                                  1

                                  2 aavxx

                                  n

                                  ii

                                  n

                                  ii

                                  a xd

                                  Projection of a Vector onto a Subspace

                                  bull Idea Member of that is Closest to

                                  (ie ldquoBest Approxrsquonrdquo)

                                  Review of Linear Algebra (Cont)x

                                  xV

                                  V

                                  Projection of a Vector onto a Subspace

                                  bull Idea Member of that is Closest to

                                  (ie ldquoBest Approxrsquonrdquo)

                                  bull Find that Solves

                                  (ldquoLeast Squaresrdquo)

                                  Review of Linear Algebra (Cont)x

                                  xV

                                  V

                                  VxPV vxVv

                                  min

                                  Projection of a Vector onto a Subspace

                                  bull Idea Member of that is Closest to

                                  (ie ldquoBest Approxrsquonrdquo)

                                  bull Find that Solves

                                  (ldquoLeast Squaresrdquo)

                                  bull For Inner Product (Hilbert) Space

                                  Exists and is Unique

                                  Review of Linear Algebra (Cont)x

                                  xV

                                  V

                                  VxPV vxVv

                                  min

                                  xPV

                                  Projection of a Vector onto a Subspace (cont)

                                  bull General Solution in for Basis Matrix

                                  Review of Linear Algebra (Cont)

                                  d VB

                                  xBBBBxP tVV

                                  tVVV

                                  1

                                  Projection of a Vector onto a Subspace (cont)

                                  bull General Solution in for Basis Matrix

                                  bull So Projrsquon Operator is Matrix Multrsquon

                                  (thus projection is another linear operation)

                                  Review of Linear Algebra (Cont)

                                  d VB

                                  xBBBBxP tVV

                                  tVVV

                                  1

                                  tVV

                                  tVVV BBBBP

                                  1

                                  Projection of a Vector onto a Subspace (cont)

                                  bull General Solution in for Basis Matrix

                                  bull So Projrsquon Operator is Matrix Multrsquon

                                  (thus projection is another linear operation)

                                  (note same operation underlies least squares)

                                  Review of Linear Algebra (Cont)

                                  d VB

                                  xBBBBxP tVV

                                  tVVV

                                  1

                                  tVV

                                  tVVV BBBBP

                                  1

                                  Review of Linear Algebra (Cont)

                                  Projection using Orthonormal Basis

                                  bull Basis Matrix is Orthonormal

                                  nnVtV IBB

                                  10

                                  01

                                  1

                                  111

                                  1

                                  1

                                  nnn

                                  n

                                  ntn

                                  t

                                  vvvv

                                  vvvv

                                  vv

                                  v

                                  v

                                  nvv 1

                                  Review of Linear Algebra (Cont)

                                  Projection using Orthonormal Basis

                                  bull Basis Matrix is Orthonormal

                                  bull So =

                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                  (Recall Right Multrsquon)

                                  nnVtV IBB

                                  10

                                  01

                                  1

                                  111

                                  1

                                  1

                                  nnn

                                  n

                                  ntn

                                  t

                                  vvvv

                                  vvvv

                                  vv

                                  v

                                  v

                                  xBBxP tVVV

                                  x V

                                  nvv 1

                                  Review of Linear Algebra (Cont)

                                  Projection using Orthonormal Basis (cont)

                                  bull For Orthogonal Complement

                                  and

                                  V

                                  xPxPx VV 222xPxPx VV

                                  Review of Linear Algebra (Cont)

                                  Projection using Orthonormal Basis (cont)

                                  bull For Orthogonal Complement

                                  and

                                  bull Parseval Inequality

                                  V

                                  xPxPx VV 222xPxPx VV

                                  2

                                  1

                                  22

                                  1

                                  22 aavxxxP

                                  n

                                  ii

                                  n

                                  iiV

                                  Review of Linear Algebra (Cont)

                                  (Real) Unitary Matrices with

                                  bull Orthonormal Basis Matrix

                                  (So All of Above Applies)

                                  ddU IUU t

                                  Review of Linear Algebra (Cont)

                                  (Real) Unitary Matrices with

                                  bull Orthonormal Basis Matrix

                                  (So All of Above Applies)

                                  bull Note Transformrsquon is Distance Preserving

                                  ddU IUU t

                                  yxdyxyxyxUyUxUdn

                                  i ii 2

                                  1

                                  Review of Linear Algebra (Cont)

                                  (Real) Unitary Matrices with

                                  bull Orthonormal Basis Matrix

                                  (So All of Above Applies)

                                  bull Note Transformrsquon is Distance Preserving

                                  bull Lin Trans (Mult by ) is ~ Rotation

                                  bull But also Includes ldquoMirror Imagesrdquo

                                  ddU IUU t

                                  yxdyxyxyxUyUxUdn

                                  i ii 2

                                  1

                                  U

                                  Review of Linear Algebra (Cont)

                                  Singular Value Decomposition (SVD)

                                  For a Matrix

                                  Find

                                  ndX

                                  Review of Linear Algebra (Cont)

                                  Singular Value Decomposition (SVD)

                                  For a Matrix

                                  Find a Diagonal Matrix

                                  with Entries

                                  called Singular Values

                                  ndX

                                  ndS

                                  )min(1 ndss

                                  Review of Linear Algebra (Cont)

                                  Singular Value Decomposition (SVD)

                                  For a Matrix

                                  Find a Diagonal Matrix

                                  with Entries

                                  called Singular Values

                                  And Unitary (Rotation) Matrices

                                  (recall )

                                  ndX

                                  ndS

                                  )min(1 ndss

                                  ddU nnV

                                  IVVUU tt

                                  Review of Linear Algebra (Cont)

                                  Singular Value Decomposition (SVD)

                                  For a Matrix

                                  Find a Diagonal Matrix

                                  with Entries

                                  called Singular Values

                                  And Unitary (Rotation) Matrices

                                  (recall )

                                  So That

                                  ndX

                                  ndS

                                  )min(1 ndss

                                  ddU nnV

                                  IVVUU tt tUSVX

                                  Review of Linear Algebra (Cont)

                                  Intuition behind Singular Value Decomposition

                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                  vVSUvVSUvX tt

                                  Review of Linear Algebra (Cont)

                                  Intuition behind Singular Value Decomposition

                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                  bull First rotate

                                  X

                                  vVSUvVSUvX tt

                                  Review of Linear Algebra (Cont)

                                  Intuition behind Singular Value Decomposition

                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                  bull First rotate

                                  bull Second rescale coordinate axes (by )

                                  X

                                  vVSUvVSUvX tt

                                  is

                                  Review of Linear Algebra (Cont)

                                  Intuition behind Singular Value Decomposition

                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                  bull First rotate

                                  bull Second rescale coordinate axes (by )

                                  bull Third rotate again

                                  X

                                  vVSUvVSUvX tt

                                  is

                                  Review of Linear Algebra (Cont)

                                  Intuition behind Singular Value Decomposition

                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                  bull First rotate

                                  bull Second rescale coordinate axes (by )

                                  bull Third rotate again

                                  bull ie have diagonalized the transformation

                                  X

                                  vVSUvVSUvX tt

                                  is

                                  SVD Compact Representation

                                  Useful Labeling

                                  Singular Values in Increasing Order

                                  Review of Linear Algebra (Cont)

                                  )min(1 dnss

                                  SVD Compact Representation

                                  Useful Labeling

                                  Singular Values in Increasing Order

                                  Note singular values = 0 can be omitted

                                  (Since do ldquo0-Stretchingrdquo)

                                  Review of Linear Algebra (Cont)

                                  )min(1 dnss

                                  Review of Linear Algebra (Cont)

                                  r

                                  SVD Compact Representation

                                  Useful Labeling

                                  Singular Values in Increasing Order

                                  Note singular values = 0 can be omitted

                                  Let = of positive singular values

                                  )min(1 dnss

                                  Review of Linear Algebra (Cont)

                                  r

                                  SVD Compact Representation

                                  Useful Labeling

                                  Singular Values in Increasing Order

                                  Note singular values = 0 can be omitted

                                  Let = of positive singular values

                                  Then

                                  Where are truncations of

                                  trnrrrd VSUX

                                  VSU

                                  )min(1 dnss

                                  Review of Linear Algebra (Cont)

                                  SVD Full Representation

                                  =

                                  Graphics Display Assumes

                                  ndX ddU ndS nn

                                  tV

                                  nd

                                  Review of Linear Algebra (Cont)

                                  SVD Full Representation

                                  =

                                  Full Rank Basis Matrix

                                  ndX ddU ndS nn

                                  tV

                                  Review of Linear Algebra (Cont)

                                  SVD Full Representation

                                  =

                                  Full Rank Basis Matrix

                                  All 0s in Bottom

                                  ndX ddU ndS nn

                                  tV

                                  Review of Linear Algebra (Cont)

                                  SVD Reduced Representation

                                  =

                                  These Columns Get 0ed Out

                                  ndX ddU nnS nn

                                  tV

                                  nnd 0

                                  Review of Linear Algebra (Cont)

                                  SVD Reduced Representation

                                  =ndX ndU nnS nn

                                  tV

                                  Review of Linear Algebra (Cont)

                                  SVD Reduced Representation

                                  =

                                  Also Some of These May be 0

                                  ndX ndU nnS nn

                                  tV

                                  Review of Linear Algebra (Cont)

                                  SVD Compact Representation

                                  = ndX rdU

                                  rrS nrtV

                                  0

                                  Review of Linear Algebra (Cont)

                                  SVD Compact Representation

                                  =

                                  These Get 0ed Out

                                  ndX rdU

                                  rrS nrtV

                                  0

                                  Review of Linear Algebra (Cont)

                                  SVD Compact Representation

                                  = ndX rdU

                                  rrS nrtV

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition

                                  For a (Symmetric) Square Matrix

                                  Find

                                  ddX

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition

                                  For a (Symmetric) Square Matrix

                                  Find a Diagonal Matrix

                                  Called Eigenvalues

                                  Convenient Ordering

                                  ddX

                                  d

                                  D

                                  0

                                  01

                                  n 1

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition

                                  For a (Symmetric) Square Matrix

                                  Find a Diagonal Matrix

                                  And an Orthonormal Matrix

                                  (ie )

                                  ddX

                                  d

                                  D

                                  0

                                  01

                                  ddB

                                  ddtt IBBBB

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition

                                  For a (Symmetric) Square Matrix

                                  Find a Diagonal Matrix

                                  And an Orthonormal Matrix

                                  (ie )

                                  So that ie

                                  ddX

                                  d

                                  D

                                  0

                                  01

                                  ddB

                                  ddtt IBBBB

                                  DBBX tBDBX

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                  (looks similar)

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                  Complex (uses )

                                  VU

                                  1i

                                  Review of Linear Algebra (Cont)

                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                  VU

                                  X

                                  BVU

                                  1i

                                  • Participant Presentations
                                  • PCA to find clusters
                                  • Statistical Smoothing
                                  • Kernel Density Estimation
                                  • Scatterplot Smoothing
                                  • Statistical Smoothing (2)
                                  • SiZer Background
                                  • SiZer Background (2)
                                  • SiZer Background (3)
                                  • SiZer Background (4)
                                  • SiZer Background (5)
                                  • SiZer Overview
                                  • PCA to find clusters (2)
                                  • PCA to find clusters (3)
                                  • PCA to find clusters (4)
                                  • PCA to find clusters (5)
                                  • PCA to find clusters (6)
                                  • PCA to find clusters (7)
                                  • Recall Yeast Cell Cycle Data
                                  • Yeast Cell Cycle Data FDA View
                                  • Yeast Cell Cycle Data FDA View (2)
                                  • Yeast Cell Cycle Data FDA View (3)
                                  • Yeast Cell Cycles Freq 2 Proj
                                  • Frequency 2 Analysis
                                  • Frequency 2 Analysis (2)
                                  • Frequency 2 Analysis (3)
                                  • Yeast Cell Cycle
                                  • SiZer Study of Distrsquon of Angles
                                  • Reclassification of Major Genes
                                  • Compare to Previous Classifrsquon
                                  • New Subpopulation View
                                  • New Subpopulation View (2)
                                  • Detailed Look at PCA
                                  • PCA Rediscovery ndash Renaming
                                  • PCA Rediscovery ndash Renaming (2)
                                  • PCA Rediscovery ndash Renaming (3)
                                  • PCA Rediscovery ndash Renaming (4)
                                  • PCA Rediscovery ndash Renaming (5)
                                  • An Interesting Historical Note
                                  • An Interesting Historical Note (2)
                                  • Detailed Look at PCA (2)
                                  • Detailed Look at PCA (3)
                                  • Review of Linear Algebra
                                  • Review of Linear Algebra (2)
                                  • Review of Linear Algebra (3)
                                  • Review of Linear Algebra (Cont)
                                  • Review of Linear Algebra (Cont) (2)
                                  • Review of Linear Algebra (Cont) (3)
                                  • Review of Linear Algebra (Cont) (4)
                                  • Review of Linear Algebra (Cont) (5)
                                  • Review of Linear Algebra (Cont) (6)
                                  • Review of Linear Algebra (Cont) (7)
                                  • Review of Linear Algebra (Cont) (8)
                                  • Review of Linear Algebra (Cont) (9)
                                  • Review of Linear Algebra (Cont) (10)
                                  • Review of Linear Algebra (Cont) (11)
                                  • Review of Linear Algebra (Cont) (12)
                                  • Review of Linear Algebra (Cont) (13)
                                  • Review of Linear Algebra (Cont) (14)
                                  • Review of Linear Algebra (Cont) (15)
                                  • Review of Linear Algebra (Cont) (16)
                                  • Review of Linear Algebra (Cont) (17)
                                  • Review of Linear Algebra (Cont) (18)
                                  • Review of Linear Algebra (Cont) (19)
                                  • Review of Linear Algebra (Cont) (20)
                                  • Review of Linear Algebra (Cont) (21)
                                  • Review of Linear Algebra (Cont) (22)
                                  • Review of Linear Algebra (Cont) (23)
                                  • Review of Linear Algebra (Cont) (24)
                                  • Review of Linear Algebra (Cont) (25)
                                  • Review of Linear Algebra (Cont) (26)
                                  • Review of Linear Algebra (Cont) (27)
                                  • Review of Linear Algebra (Cont) (28)
                                  • Review of Linear Algebra (Cont) (29)
                                  • Review of Linear Algebra (Cont) (30)
                                  • Review of Linear Algebra (Cont) (31)
                                  • Review of Linear Algebra (Cont) (32)
                                  • Review of Linear Algebra (Cont) (33)
                                  • Review of Linear Algebra (Cont) (34)
                                  • Review of Linear Algebra (Cont) (35)
                                  • Review of Linear Algebra (Cont) (36)
                                  • Review of Linear Algebra (Cont) (37)
                                  • Review of Linear Algebra (Cont) (38)
                                  • Review of Linear Algebra (Cont) (39)
                                  • Review of Linear Algebra (Cont) (40)
                                  • Review of Linear Algebra (Cont) (41)
                                  • Review of Linear Algebra (Cont) (42)
                                  • Review of Linear Algebra (Cont) (43)
                                  • Review of Linear Algebra (Cont) (44)
                                  • Review of Linear Algebra (Cont) (45)
                                  • Review of Linear Algebra (Cont) (46)
                                  • Review of Linear Algebra (Cont) (47)
                                  • Review of Linear Algebra (Cont) (48)
                                  • Review of Linear Algebra (Cont) (49)
                                  • Review of Linear Algebra (Cont) (50)
                                  • Review of Linear Algebra (Cont) (51)
                                  • Review of Linear Algebra (Cont) (52)
                                  • Review of Linear Algebra (Cont) (53)
                                  • Review of Linear Algebra (Cont) (54)
                                  • Review of Linear Algebra (Cont) (55)
                                  • Review of Linear Algebra (Cont) (56)
                                  • Review of Linear Algebra (Cont) (57)
                                  • Review of Linear Algebra (Cont) (58)
                                  • Review of Linear Algebra (Cont) (59)
                                  • Review of Linear Algebra (Cont) (60)
                                  • Review of Linear Algebra (Cont) (61)
                                  • Review of Linear Algebra (Cont) (62)
                                  • Review of Linear Algebra (Cont) (63)
                                  • Review of Linear Algebra (Cont) (64)
                                  • Review of Linear Algebra (Cont) (65)
                                  • Review of Linear Algebra (Cont) (66)
                                  • Review of Linear Algebra (Cont) (67)
                                  • Review of Linear Algebra (Cont) (68)
                                  • Review of Linear Algebra (Cont) (69)
                                  • Review of Linear Algebra (Cont) (70)
                                  • Review of Linear Algebra (Cont) (71)
                                  • Review of Linear Algebra (Cont) (72)
                                  • Review of Linear Algebra (Cont) (73)
                                  • Review of Linear Algebra (Cont) (74)
                                  • Review of Linear Algebra (Cont) (75)
                                  • Review of Linear Algebra (Cont) (76)
                                  • Review of Linear Algebra (Cont) (77)
                                  • Review of Linear Algebra (Cont) (78)
                                  • Review of Linear Algebra (Cont) (79)
                                  • Review of Linear Algebra (Cont) (80)
                                  • Review of Linear Algebra (Cont) (81)

                                    PCA to find clustersSiZer analysis of Mass Flux PC1

                                    Conclusion

                                    bull Found 3 significant clusters

                                    bull Correspond to 3 known ldquocloud typesrdquo

                                    bull Worth deeper investigation

                                    Recall Yeast Cell Cycle Data

                                    bull ldquoGene Expressionrdquo ndash Micro-array data

                                    bull Data (after major preprocessing) Expression ldquolevelrdquo of

                                    bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                                    10s)bull Interesting statistical issue

                                    High Dimension Low Sample Size data

                                    (HDLSS)

                                    Yeast Cell Cycle Data FDA View

                                    Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                                    Yeast Cell Cycle Data FDA View

                                    Periodic genes

                                    Naiumlve

                                    approach

                                    Simple PCA

                                    Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                    ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                    ldquovariationrdquobull PCA finds ldquodirections of maximal

                                    variationrdquobull Often but not always same as

                                    ldquointeresting directionsrdquobull Here need better approach to study

                                    periodicities

                                    Yeast Cell Cycles Freq 2 Proj

                                    PCA on

                                    Freq 2

                                    Periodic

                                    Component

                                    Of Data

                                    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                    Approach from Zhao Marron amp Wells (2004)

                                    Frequency 2 Analysis

                                    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                    classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                    notable differencesbull Now try to improve ldquophase classificationrdquo

                                    Yeast Cell CycleRevisit ldquophase classificationrdquo

                                    approachbull Use outer 200 genes

                                    (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                    (finds significant bumps etc in histogram)

                                    bull Carefully redrew boundariesbull Check by studying kde angles

                                    SiZer Study of Distrsquon of Angles

                                    Reclassification of Major Genes

                                    Compare to Previous Classifrsquon

                                    New Subpopulation View

                                    New Subpopulation View

                                    NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                    Detailed Look at PCA

                                    Now Study ldquoFolklorerdquo More Carefully

                                    bull BackGround

                                    bull History

                                    bull Underpinnings

                                    (Mathematical amp Computational)

                                    Good Overall Reference Jolliffe (2002)

                                    PCA Rediscovery ndash Renaming

                                    Statistics Principal Component Analysis (PCA)

                                    PCA Rediscovery ndash Renaming

                                    Statistics Principal Component Analysis (PCA)

                                    Social Sciences Factor Analysis (PCA is a subset)

                                    PCA Rediscovery ndash Renaming

                                    Statistics Principal Component Analysis (PCA)

                                    Social Sciences Factor Analysis (PCA is a subset)

                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                    PCA Rediscovery ndash Renaming

                                    Statistics Principal Component Analysis (PCA)

                                    Social Sciences Factor Analysis (PCA is a subset)

                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                    PCA Rediscovery ndash Renaming

                                    Statistics Principal Component Analysis (PCA)

                                    Social Sciences Factor Analysis (PCA is a subset)

                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                    Geo-Sciences Empirical Orthogonal Functions (EOF)

                                    An Interesting Historical Note

                                    The 1st () application of PCA to Functional

                                    Data Analysis

                                    An Interesting Historical Note

                                    The 1st () application of PCA to Functional

                                    Data Analysis

                                    Rao (1958)

                                    1st Paper with ldquoCurves as Data Objectsrdquo

                                    viewpoint

                                    Detailed Look at PCA

                                    Three Important (amp Interesting) Viewpoints

                                    1 Mathematics

                                    2 Numerics

                                    3 Statistics

                                    Goal Study Interrelationships

                                    Detailed Look at PCA

                                    Three Important (amp Interesting) Viewpoints

                                    1 Mathematics

                                    2 Numerics

                                    3 Statistics

                                    1st Review Linear Alg and Multivar Prob

                                    Review of Linear Algebra

                                    Vector Space

                                    bull set of ldquovectorsrdquo

                                    bull and ldquoscalarsrdquo (coefficients)

                                    xa

                                    Review of Linear Algebra

                                    Vector Space

                                    bull set of ldquovectorsrdquo

                                    bull and ldquoscalarsrdquo (coefficients)

                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                    ( in space)

                                    xa

                                    i

                                    ii xa

                                    Review of Linear Algebra

                                    Vector Space

                                    bull set of ldquovectorsrdquo

                                    bull and ldquoscalarsrdquo (coefficients)

                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                    ( in space)

                                    eg

                                    ldquo dim Euclidrsquon spacerdquo

                                    xa

                                    i

                                    ii xa

                                    d

                                    d

                                    d xx

                                    x

                                    x

                                    x 1

                                    1

                                    d

                                    Review of Linear Algebra (Cont)

                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                    Review of Linear Algebra (Cont)

                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                    Review of Linear Algebra (Cont)

                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                    Note Planes not Through the Origin

                                    are not Subspaces

                                    (Do not Contain )00 x

                                    Review of Linear Algebra (Cont)

                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                    (all Linear Combos of them =

                                    = Containing Hyperplane

                                    through Origin)

                                    Review of Linear Algebra (Cont)

                                    Basis of Subspace Set of Vectors that

                                    bull Span ie Everything is a Lin Com of them

                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                    Review of Linear Algebra (Cont)

                                    Basis of Subspace Set of Vectors that

                                    bull Span ie Everything is a Lin Com of them

                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                    bull eg ldquoUnit Vector Basisrdquo

                                    bull Since

                                    d

                                    1

                                    0

                                    0

                                    0

                                    1

                                    0

                                    0

                                    0

                                    1

                                    1

                                    0

                                    0

                                    0

                                    1

                                    0

                                    0

                                    0

                                    1

                                    212

                                    1

                                    d

                                    d

                                    xxx

                                    x

                                    x

                                    x

                                    Review of Linear Algebra (Cont)

                                    Basis Matrix of subspace of

                                    Given a basis

                                    create matrix of columns

                                    dnvv 1

                                    nddnd

                                    n

                                    n

                                    vv

                                    vv

                                    vvB

                                    1

                                    111

                                    1

                                    Review of Linear Algebra (Cont)

                                    Then linear combo is a matrix multiplicatrsquon

                                    where

                                    n

                                    iii aBva

                                    1

                                    na

                                    a

                                    a 1

                                    Review of Linear Algebra (Cont)

                                    Then linear combo is a matrix multiplicatrsquon

                                    where

                                    Note Right Multiplication Gives

                                    Linear Combination of Column Vectors

                                    n

                                    iii aBva

                                    1

                                    na

                                    a

                                    a 1

                                    Review of Linear Algebra (Cont)

                                    Then linear combo is a matrix multiplicatrsquon

                                    where

                                    Check sizes

                                    n

                                    iii aBva

                                    1

                                    na

                                    a

                                    a 1

                                    )1()(1 nndd

                                    Review of Linear Algebra (Cont)

                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                    For matrices

                                    Define the Matrix Product

                                    mkk

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    nmm

                                    n

                                    bb

                                    bb

                                    B

                                    1

                                    111

                                    m

                                    iniik

                                    m

                                    iiik

                                    m

                                    inii

                                    m

                                    iii

                                    baba

                                    baba

                                    AB

                                    1

                                    11

                                    11

                                    111

                                    Review of Linear Algebra (Cont)

                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                    For matrices

                                    Define the Matrix Product

                                    (Inner Products of Rows With Columns )

                                    mkk

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    nmm

                                    n

                                    bb

                                    bb

                                    B

                                    1

                                    111

                                    m

                                    iniik

                                    m

                                    iiik

                                    m

                                    inii

                                    m

                                    iii

                                    baba

                                    baba

                                    AB

                                    1

                                    11

                                    11

                                    111

                                    A B

                                    Review of Linear Algebra (Cont)

                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                    For matrices

                                    Define the Matrix Product

                                    (Inner Products of Rows With Columns )

                                    (Composition of Linear Transformations)

                                    mkk

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    nmm

                                    n

                                    bb

                                    bb

                                    B

                                    1

                                    111

                                    m

                                    iniik

                                    m

                                    iiik

                                    m

                                    inii

                                    m

                                    iii

                                    baba

                                    baba

                                    AB

                                    1

                                    11

                                    11

                                    111

                                    A B

                                    Review of Linear Algebra (Cont)

                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                    For matrices

                                    Define the Matrix Product

                                    (Inner Products of Rows With Columns )

                                    (Composition of Linear Transformations)

                                    Often Useful to Check Sizes

                                    mkk

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    nmm

                                    n

                                    bb

                                    bb

                                    B

                                    1

                                    111

                                    m

                                    iniik

                                    m

                                    iiik

                                    m

                                    inii

                                    m

                                    iii

                                    baba

                                    baba

                                    AB

                                    1

                                    11

                                    11

                                    111

                                    nmmknk

                                    A B

                                    Review of Linear Algebra (Cont)

                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                    For matrices

                                    Define the Matrix Product

                                    (Inner Products of Rows With Columns )

                                    (Composition of Linear Transformations)

                                    Often Useful to Check Sizes

                                    mkk

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    nmm

                                    n

                                    bb

                                    bb

                                    B

                                    1

                                    111

                                    m

                                    iniik

                                    m

                                    iiik

                                    m

                                    inii

                                    m

                                    iii

                                    baba

                                    baba

                                    AB

                                    1

                                    11

                                    11

                                    111

                                    nmmknk

                                    A B

                                    Review of Linear Algebra (Cont)

                                    Matrix Trace

                                    bull For a Square Matrix

                                    bull Define

                                    mmm

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    m

                                    iiiaAtr

                                    1)(

                                    Review of Linear Algebra (Cont)

                                    Matrix Trace

                                    bull For a Square Matrix

                                    bull Define

                                    bull Trace Commutes with Matrix Multiplication

                                    mmm

                                    m

                                    aa

                                    aa

                                    A

                                    1

                                    111

                                    m

                                    iiiaAtr

                                    1)(

                                    BAtrABtr

                                    Review of Linear Algebra (Cont)

                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                    bull Number of Elements in a Basis (Unique)

                                    Review of Linear Algebra (Cont)

                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                    bull Number of Elements in a Basis (Unique)

                                    bull (Use Basis Above)

                                    bull eg dim of a line is 1

                                    bull eg dim of a plane is 2

                                    dd dim

                                    Review of Linear Algebra (Cont)

                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                    bull Number of Elements in a Basis (Unique)

                                    bull (Use Basis Above)

                                    bull eg dim of a line is 1

                                    bull eg dim of a plane is 2

                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                    (in Statistical Uses eg ANOVA)

                                    dd dim

                                    Review of Linear Algebra (Cont)

                                    Norm of a Vector

                                    bull in d 21

                                    21

                                    1

                                    2 xxxx td

                                    jj

                                    Review of Linear Algebra (Cont)

                                    Norm of a Vector

                                    bull in

                                    bull Idea length of the vector

                                    d 2121

                                    1

                                    2 xxxx td

                                    jj

                                    Review of Linear Algebra (Cont)

                                    Norm of a Vector

                                    bull in

                                    bull Idea length of the vector

                                    bull Note strange properties for high

                                    eg ldquolength of diagonal of unit cuberdquo =

                                    d 2121

                                    1

                                    2 xxxx td

                                    jj

                                    d

                                    d

                                    Review of Linear Algebra (Cont)

                                    Norm of a Vector (cont)

                                    bull Length Normalized Vector

                                    (has Length 1 thus on Surf of Unit Sphere

                                    amp is a Direction Vector)

                                    x

                                    x

                                    Review of Linear Algebra (Cont)

                                    Norm of a Vector (cont)

                                    bull Length Normalized Vector

                                    (has Length 1 thus on Surf of Unit Sphere

                                    amp is a Direction Vector)

                                    bull Define Distance as

                                    x

                                    x

                                    yxyxyxyxd t

                                    Review of Linear Algebra (Cont)

                                    Inner (Dot Scalar) Product

                                    bull for Vectors and

                                    yxyxyx td

                                    jjj

                                    1

                                    x y

                                    Review of Linear Algebra (Cont)

                                    Inner (Dot Scalar) Product

                                    bull for Vectors and

                                    bull Related to Norm via

                                    yxyxyx td

                                    jjj

                                    1

                                    x y

                                    21 xxx

                                    Review of Linear Algebra (Cont)

                                    Inner (Dot Scalar) Product (cont)

                                    bull measures ldquoangle between and rdquo as

                                    yyxx

                                    yx

                                    yx

                                    yxyxangle

                                    tt

                                    t

                                    11 cos

                                    cos

                                    x y

                                    Review of Linear Algebra (Cont)

                                    Inner (Dot Scalar) Product (cont)

                                    bull measures ldquoangle between and rdquo as

                                    bull key to Orthogonality ie Perpendiculrsquoty

                                    if and only if

                                    yyxx

                                    yx

                                    yx

                                    yxyxangle

                                    tt

                                    t

                                    11 cos

                                    cos

                                    x y

                                    yx 0 yx

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis

                                    bull All Orthogonal to each other

                                    ie for

                                    bull All have Length 1

                                    ie for

                                    nvv 1

                                    1 ii vv

                                    0 ii vv ii

                                    ni 1

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis (cont)

                                    bull Spectral Representation

                                    where

                                    nvv 1

                                    n

                                    iii vax

                                    1

                                    ii vxa

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis (cont)

                                    bull Spectral Representation

                                    where

                                    (Coefficient is Inner Product Cool Notation)

                                    nvv 1

                                    n

                                    iii vax

                                    1

                                    ii vxa

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis (cont)

                                    bull Spectral Representation

                                    where

                                    Check

                                    nvv 1

                                    n

                                    iii vax

                                    1

                                    ii vxa

                                    iii

                                    n

                                    iii

                                    n

                                    iiii avvavvavx

                                    1

                                    1

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis (cont)

                                    bull Spectral Representation

                                    where

                                    Check

                                    bull Matrix Notation where ie

                                    For the Basis Matrix

                                    nvv 1

                                    n

                                    iii vax

                                    1

                                    ii vxa

                                    iii

                                    n

                                    iii

                                    n

                                    iiii avvavvavx

                                    1

                                    1

                                    aBx Bxa tt xBa t

                                    nvvB 1

                                    Review of Linear Algebra (Cont)

                                    Orthonormal Basis (cont)

                                    bull Spectral Representation

                                    where

                                    Check

                                    bull Matrix Notation where ie

                                    is called transform of

                                    (eg Fourier or Wavelet)

                                    nvv 1

                                    n

                                    iii vax

                                    1

                                    ii vxa

                                    iii

                                    n

                                    iii

                                    n

                                    iiii avvavvavx

                                    1

                                    1

                                    aBx Bxa tt xBa t

                                    xa

                                    Review of Linear Algebra (Cont)

                                    Parseval identity for

                                    in subsp genrsquod by o n basis

                                    x

                                    nvv 1

                                    2

                                    1

                                    22

                                    1

                                    2 aavxx

                                    n

                                    ii

                                    n

                                    ii

                                    Review of Linear Algebra (Cont)

                                    Parseval identity for

                                    in subsp genrsquod by o n basis

                                    bull Pythagorean theorem

                                    bull ldquoDecomposition of Energyrdquo

                                    bull ANOVA - sums of squares

                                    x

                                    nvv 1

                                    2

                                    1

                                    22

                                    1

                                    2 aavxx

                                    n

                                    ii

                                    n

                                    ii

                                    Review of Linear Algebra (Cont)

                                    Parseval identity for

                                    in subsp genrsquod by o n basis

                                    bull Pythagorean theorem

                                    bull ldquoDecomposition of Energyrdquo

                                    bull ANOVA - sums of squares

                                    bull Transform has same length as

                                    ie ldquorotation in rdquo

                                    x

                                    nvv 1

                                    2

                                    1

                                    22

                                    1

                                    2 aavxx

                                    n

                                    ii

                                    n

                                    ii

                                    a xd

                                    Projection of a Vector onto a Subspace

                                    bull Idea Member of that is Closest to

                                    (ie ldquoBest Approxrsquonrdquo)

                                    Review of Linear Algebra (Cont)x

                                    xV

                                    V

                                    Projection of a Vector onto a Subspace

                                    bull Idea Member of that is Closest to

                                    (ie ldquoBest Approxrsquonrdquo)

                                    bull Find that Solves

                                    (ldquoLeast Squaresrdquo)

                                    Review of Linear Algebra (Cont)x

                                    xV

                                    V

                                    VxPV vxVv

                                    min

                                    Projection of a Vector onto a Subspace

                                    bull Idea Member of that is Closest to

                                    (ie ldquoBest Approxrsquonrdquo)

                                    bull Find that Solves

                                    (ldquoLeast Squaresrdquo)

                                    bull For Inner Product (Hilbert) Space

                                    Exists and is Unique

                                    Review of Linear Algebra (Cont)x

                                    xV

                                    V

                                    VxPV vxVv

                                    min

                                    xPV

                                    Projection of a Vector onto a Subspace (cont)

                                    bull General Solution in for Basis Matrix

                                    Review of Linear Algebra (Cont)

                                    d VB

                                    xBBBBxP tVV

                                    tVVV

                                    1

                                    Projection of a Vector onto a Subspace (cont)

                                    bull General Solution in for Basis Matrix

                                    bull So Projrsquon Operator is Matrix Multrsquon

                                    (thus projection is another linear operation)

                                    Review of Linear Algebra (Cont)

                                    d VB

                                    xBBBBxP tVV

                                    tVVV

                                    1

                                    tVV

                                    tVVV BBBBP

                                    1

                                    Projection of a Vector onto a Subspace (cont)

                                    bull General Solution in for Basis Matrix

                                    bull So Projrsquon Operator is Matrix Multrsquon

                                    (thus projection is another linear operation)

                                    (note same operation underlies least squares)

                                    Review of Linear Algebra (Cont)

                                    d VB

                                    xBBBBxP tVV

                                    tVVV

                                    1

                                    tVV

                                    tVVV BBBBP

                                    1

                                    Review of Linear Algebra (Cont)

                                    Projection using Orthonormal Basis

                                    bull Basis Matrix is Orthonormal

                                    nnVtV IBB

                                    10

                                    01

                                    1

                                    111

                                    1

                                    1

                                    nnn

                                    n

                                    ntn

                                    t

                                    vvvv

                                    vvvv

                                    vv

                                    v

                                    v

                                    nvv 1

                                    Review of Linear Algebra (Cont)

                                    Projection using Orthonormal Basis

                                    bull Basis Matrix is Orthonormal

                                    bull So =

                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                    (Recall Right Multrsquon)

                                    nnVtV IBB

                                    10

                                    01

                                    1

                                    111

                                    1

                                    1

                                    nnn

                                    n

                                    ntn

                                    t

                                    vvvv

                                    vvvv

                                    vv

                                    v

                                    v

                                    xBBxP tVVV

                                    x V

                                    nvv 1

                                    Review of Linear Algebra (Cont)

                                    Projection using Orthonormal Basis (cont)

                                    bull For Orthogonal Complement

                                    and

                                    V

                                    xPxPx VV 222xPxPx VV

                                    Review of Linear Algebra (Cont)

                                    Projection using Orthonormal Basis (cont)

                                    bull For Orthogonal Complement

                                    and

                                    bull Parseval Inequality

                                    V

                                    xPxPx VV 222xPxPx VV

                                    2

                                    1

                                    22

                                    1

                                    22 aavxxxP

                                    n

                                    ii

                                    n

                                    iiV

                                    Review of Linear Algebra (Cont)

                                    (Real) Unitary Matrices with

                                    bull Orthonormal Basis Matrix

                                    (So All of Above Applies)

                                    ddU IUU t

                                    Review of Linear Algebra (Cont)

                                    (Real) Unitary Matrices with

                                    bull Orthonormal Basis Matrix

                                    (So All of Above Applies)

                                    bull Note Transformrsquon is Distance Preserving

                                    ddU IUU t

                                    yxdyxyxyxUyUxUdn

                                    i ii 2

                                    1

                                    Review of Linear Algebra (Cont)

                                    (Real) Unitary Matrices with

                                    bull Orthonormal Basis Matrix

                                    (So All of Above Applies)

                                    bull Note Transformrsquon is Distance Preserving

                                    bull Lin Trans (Mult by ) is ~ Rotation

                                    bull But also Includes ldquoMirror Imagesrdquo

                                    ddU IUU t

                                    yxdyxyxyxUyUxUdn

                                    i ii 2

                                    1

                                    U

                                    Review of Linear Algebra (Cont)

                                    Singular Value Decomposition (SVD)

                                    For a Matrix

                                    Find

                                    ndX

                                    Review of Linear Algebra (Cont)

                                    Singular Value Decomposition (SVD)

                                    For a Matrix

                                    Find a Diagonal Matrix

                                    with Entries

                                    called Singular Values

                                    ndX

                                    ndS

                                    )min(1 ndss

                                    Review of Linear Algebra (Cont)

                                    Singular Value Decomposition (SVD)

                                    For a Matrix

                                    Find a Diagonal Matrix

                                    with Entries

                                    called Singular Values

                                    And Unitary (Rotation) Matrices

                                    (recall )

                                    ndX

                                    ndS

                                    )min(1 ndss

                                    ddU nnV

                                    IVVUU tt

                                    Review of Linear Algebra (Cont)

                                    Singular Value Decomposition (SVD)

                                    For a Matrix

                                    Find a Diagonal Matrix

                                    with Entries

                                    called Singular Values

                                    And Unitary (Rotation) Matrices

                                    (recall )

                                    So That

                                    ndX

                                    ndS

                                    )min(1 ndss

                                    ddU nnV

                                    IVVUU tt tUSVX

                                    Review of Linear Algebra (Cont)

                                    Intuition behind Singular Value Decomposition

                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                    vVSUvVSUvX tt

                                    Review of Linear Algebra (Cont)

                                    Intuition behind Singular Value Decomposition

                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                    bull First rotate

                                    X

                                    vVSUvVSUvX tt

                                    Review of Linear Algebra (Cont)

                                    Intuition behind Singular Value Decomposition

                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                    bull First rotate

                                    bull Second rescale coordinate axes (by )

                                    X

                                    vVSUvVSUvX tt

                                    is

                                    Review of Linear Algebra (Cont)

                                    Intuition behind Singular Value Decomposition

                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                    bull First rotate

                                    bull Second rescale coordinate axes (by )

                                    bull Third rotate again

                                    X

                                    vVSUvVSUvX tt

                                    is

                                    Review of Linear Algebra (Cont)

                                    Intuition behind Singular Value Decomposition

                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                    bull First rotate

                                    bull Second rescale coordinate axes (by )

                                    bull Third rotate again

                                    bull ie have diagonalized the transformation

                                    X

                                    vVSUvVSUvX tt

                                    is

                                    SVD Compact Representation

                                    Useful Labeling

                                    Singular Values in Increasing Order

                                    Review of Linear Algebra (Cont)

                                    )min(1 dnss

                                    SVD Compact Representation

                                    Useful Labeling

                                    Singular Values in Increasing Order

                                    Note singular values = 0 can be omitted

                                    (Since do ldquo0-Stretchingrdquo)

                                    Review of Linear Algebra (Cont)

                                    )min(1 dnss

                                    Review of Linear Algebra (Cont)

                                    r

                                    SVD Compact Representation

                                    Useful Labeling

                                    Singular Values in Increasing Order

                                    Note singular values = 0 can be omitted

                                    Let = of positive singular values

                                    )min(1 dnss

                                    Review of Linear Algebra (Cont)

                                    r

                                    SVD Compact Representation

                                    Useful Labeling

                                    Singular Values in Increasing Order

                                    Note singular values = 0 can be omitted

                                    Let = of positive singular values

                                    Then

                                    Where are truncations of

                                    trnrrrd VSUX

                                    VSU

                                    )min(1 dnss

                                    Review of Linear Algebra (Cont)

                                    SVD Full Representation

                                    =

                                    Graphics Display Assumes

                                    ndX ddU ndS nn

                                    tV

                                    nd

                                    Review of Linear Algebra (Cont)

                                    SVD Full Representation

                                    =

                                    Full Rank Basis Matrix

                                    ndX ddU ndS nn

                                    tV

                                    Review of Linear Algebra (Cont)

                                    SVD Full Representation

                                    =

                                    Full Rank Basis Matrix

                                    All 0s in Bottom

                                    ndX ddU ndS nn

                                    tV

                                    Review of Linear Algebra (Cont)

                                    SVD Reduced Representation

                                    =

                                    These Columns Get 0ed Out

                                    ndX ddU nnS nn

                                    tV

                                    nnd 0

                                    Review of Linear Algebra (Cont)

                                    SVD Reduced Representation

                                    =ndX ndU nnS nn

                                    tV

                                    Review of Linear Algebra (Cont)

                                    SVD Reduced Representation

                                    =

                                    Also Some of These May be 0

                                    ndX ndU nnS nn

                                    tV

                                    Review of Linear Algebra (Cont)

                                    SVD Compact Representation

                                    = ndX rdU

                                    rrS nrtV

                                    0

                                    Review of Linear Algebra (Cont)

                                    SVD Compact Representation

                                    =

                                    These Get 0ed Out

                                    ndX rdU

                                    rrS nrtV

                                    0

                                    Review of Linear Algebra (Cont)

                                    SVD Compact Representation

                                    = ndX rdU

                                    rrS nrtV

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition

                                    For a (Symmetric) Square Matrix

                                    Find

                                    ddX

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition

                                    For a (Symmetric) Square Matrix

                                    Find a Diagonal Matrix

                                    Called Eigenvalues

                                    Convenient Ordering

                                    ddX

                                    d

                                    D

                                    0

                                    01

                                    n 1

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition

                                    For a (Symmetric) Square Matrix

                                    Find a Diagonal Matrix

                                    And an Orthonormal Matrix

                                    (ie )

                                    ddX

                                    d

                                    D

                                    0

                                    01

                                    ddB

                                    ddtt IBBBB

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition

                                    For a (Symmetric) Square Matrix

                                    Find a Diagonal Matrix

                                    And an Orthonormal Matrix

                                    (ie )

                                    So that ie

                                    ddX

                                    d

                                    D

                                    0

                                    01

                                    ddB

                                    ddtt IBBBB

                                    DBBX tBDBX

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                    (looks similar)

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                    Complex (uses )

                                    VU

                                    1i

                                    Review of Linear Algebra (Cont)

                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                    VU

                                    X

                                    BVU

                                    1i

                                    • Participant Presentations
                                    • PCA to find clusters
                                    • Statistical Smoothing
                                    • Kernel Density Estimation
                                    • Scatterplot Smoothing
                                    • Statistical Smoothing (2)
                                    • SiZer Background
                                    • SiZer Background (2)
                                    • SiZer Background (3)
                                    • SiZer Background (4)
                                    • SiZer Background (5)
                                    • SiZer Overview
                                    • PCA to find clusters (2)
                                    • PCA to find clusters (3)
                                    • PCA to find clusters (4)
                                    • PCA to find clusters (5)
                                    • PCA to find clusters (6)
                                    • PCA to find clusters (7)
                                    • Recall Yeast Cell Cycle Data
                                    • Yeast Cell Cycle Data FDA View
                                    • Yeast Cell Cycle Data FDA View (2)
                                    • Yeast Cell Cycle Data FDA View (3)
                                    • Yeast Cell Cycles Freq 2 Proj
                                    • Frequency 2 Analysis
                                    • Frequency 2 Analysis (2)
                                    • Frequency 2 Analysis (3)
                                    • Yeast Cell Cycle
                                    • SiZer Study of Distrsquon of Angles
                                    • Reclassification of Major Genes
                                    • Compare to Previous Classifrsquon
                                    • New Subpopulation View
                                    • New Subpopulation View (2)
                                    • Detailed Look at PCA
                                    • PCA Rediscovery ndash Renaming
                                    • PCA Rediscovery ndash Renaming (2)
                                    • PCA Rediscovery ndash Renaming (3)
                                    • PCA Rediscovery ndash Renaming (4)
                                    • PCA Rediscovery ndash Renaming (5)
                                    • An Interesting Historical Note
                                    • An Interesting Historical Note (2)
                                    • Detailed Look at PCA (2)
                                    • Detailed Look at PCA (3)
                                    • Review of Linear Algebra
                                    • Review of Linear Algebra (2)
                                    • Review of Linear Algebra (3)
                                    • Review of Linear Algebra (Cont)
                                    • Review of Linear Algebra (Cont) (2)
                                    • Review of Linear Algebra (Cont) (3)
                                    • Review of Linear Algebra (Cont) (4)
                                    • Review of Linear Algebra (Cont) (5)
                                    • Review of Linear Algebra (Cont) (6)
                                    • Review of Linear Algebra (Cont) (7)
                                    • Review of Linear Algebra (Cont) (8)
                                    • Review of Linear Algebra (Cont) (9)
                                    • Review of Linear Algebra (Cont) (10)
                                    • Review of Linear Algebra (Cont) (11)
                                    • Review of Linear Algebra (Cont) (12)
                                    • Review of Linear Algebra (Cont) (13)
                                    • Review of Linear Algebra (Cont) (14)
                                    • Review of Linear Algebra (Cont) (15)
                                    • Review of Linear Algebra (Cont) (16)
                                    • Review of Linear Algebra (Cont) (17)
                                    • Review of Linear Algebra (Cont) (18)
                                    • Review of Linear Algebra (Cont) (19)
                                    • Review of Linear Algebra (Cont) (20)
                                    • Review of Linear Algebra (Cont) (21)
                                    • Review of Linear Algebra (Cont) (22)
                                    • Review of Linear Algebra (Cont) (23)
                                    • Review of Linear Algebra (Cont) (24)
                                    • Review of Linear Algebra (Cont) (25)
                                    • Review of Linear Algebra (Cont) (26)
                                    • Review of Linear Algebra (Cont) (27)
                                    • Review of Linear Algebra (Cont) (28)
                                    • Review of Linear Algebra (Cont) (29)
                                    • Review of Linear Algebra (Cont) (30)
                                    • Review of Linear Algebra (Cont) (31)
                                    • Review of Linear Algebra (Cont) (32)
                                    • Review of Linear Algebra (Cont) (33)
                                    • Review of Linear Algebra (Cont) (34)
                                    • Review of Linear Algebra (Cont) (35)
                                    • Review of Linear Algebra (Cont) (36)
                                    • Review of Linear Algebra (Cont) (37)
                                    • Review of Linear Algebra (Cont) (38)
                                    • Review of Linear Algebra (Cont) (39)
                                    • Review of Linear Algebra (Cont) (40)
                                    • Review of Linear Algebra (Cont) (41)
                                    • Review of Linear Algebra (Cont) (42)
                                    • Review of Linear Algebra (Cont) (43)
                                    • Review of Linear Algebra (Cont) (44)
                                    • Review of Linear Algebra (Cont) (45)
                                    • Review of Linear Algebra (Cont) (46)
                                    • Review of Linear Algebra (Cont) (47)
                                    • Review of Linear Algebra (Cont) (48)
                                    • Review of Linear Algebra (Cont) (49)
                                    • Review of Linear Algebra (Cont) (50)
                                    • Review of Linear Algebra (Cont) (51)
                                    • Review of Linear Algebra (Cont) (52)
                                    • Review of Linear Algebra (Cont) (53)
                                    • Review of Linear Algebra (Cont) (54)
                                    • Review of Linear Algebra (Cont) (55)
                                    • Review of Linear Algebra (Cont) (56)
                                    • Review of Linear Algebra (Cont) (57)
                                    • Review of Linear Algebra (Cont) (58)
                                    • Review of Linear Algebra (Cont) (59)
                                    • Review of Linear Algebra (Cont) (60)
                                    • Review of Linear Algebra (Cont) (61)
                                    • Review of Linear Algebra (Cont) (62)
                                    • Review of Linear Algebra (Cont) (63)
                                    • Review of Linear Algebra (Cont) (64)
                                    • Review of Linear Algebra (Cont) (65)
                                    • Review of Linear Algebra (Cont) (66)
                                    • Review of Linear Algebra (Cont) (67)
                                    • Review of Linear Algebra (Cont) (68)
                                    • Review of Linear Algebra (Cont) (69)
                                    • Review of Linear Algebra (Cont) (70)
                                    • Review of Linear Algebra (Cont) (71)
                                    • Review of Linear Algebra (Cont) (72)
                                    • Review of Linear Algebra (Cont) (73)
                                    • Review of Linear Algebra (Cont) (74)
                                    • Review of Linear Algebra (Cont) (75)
                                    • Review of Linear Algebra (Cont) (76)
                                    • Review of Linear Algebra (Cont) (77)
                                    • Review of Linear Algebra (Cont) (78)
                                    • Review of Linear Algebra (Cont) (79)
                                    • Review of Linear Algebra (Cont) (80)
                                    • Review of Linear Algebra (Cont) (81)

                                      Recall Yeast Cell Cycle Data

                                      bull ldquoGene Expressionrdquo ndash Micro-array data

                                      bull Data (after major preprocessing) Expression ldquolevelrdquo of

                                      bull thousands of genes (d ~ 1000s)bull but only dozens of ldquocasesrdquo (n ~

                                      10s)bull Interesting statistical issue

                                      High Dimension Low Sample Size data

                                      (HDLSS)

                                      Yeast Cell Cycle Data FDA View

                                      Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                                      Yeast Cell Cycle Data FDA View

                                      Periodic genes

                                      Naiumlve

                                      approach

                                      Simple PCA

                                      Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                      ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                      ldquovariationrdquobull PCA finds ldquodirections of maximal

                                      variationrdquobull Often but not always same as

                                      ldquointeresting directionsrdquobull Here need better approach to study

                                      periodicities

                                      Yeast Cell Cycles Freq 2 Proj

                                      PCA on

                                      Freq 2

                                      Periodic

                                      Component

                                      Of Data

                                      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                      Approach from Zhao Marron amp Wells (2004)

                                      Frequency 2 Analysis

                                      Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                      cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                      classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                      notable differencesbull Now try to improve ldquophase classificationrdquo

                                      Yeast Cell CycleRevisit ldquophase classificationrdquo

                                      approachbull Use outer 200 genes

                                      (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                      (finds significant bumps etc in histogram)

                                      bull Carefully redrew boundariesbull Check by studying kde angles

                                      SiZer Study of Distrsquon of Angles

                                      Reclassification of Major Genes

                                      Compare to Previous Classifrsquon

                                      New Subpopulation View

                                      New Subpopulation View

                                      NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                      Detailed Look at PCA

                                      Now Study ldquoFolklorerdquo More Carefully

                                      bull BackGround

                                      bull History

                                      bull Underpinnings

                                      (Mathematical amp Computational)

                                      Good Overall Reference Jolliffe (2002)

                                      PCA Rediscovery ndash Renaming

                                      Statistics Principal Component Analysis (PCA)

                                      PCA Rediscovery ndash Renaming

                                      Statistics Principal Component Analysis (PCA)

                                      Social Sciences Factor Analysis (PCA is a subset)

                                      PCA Rediscovery ndash Renaming

                                      Statistics Principal Component Analysis (PCA)

                                      Social Sciences Factor Analysis (PCA is a subset)

                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                      PCA Rediscovery ndash Renaming

                                      Statistics Principal Component Analysis (PCA)

                                      Social Sciences Factor Analysis (PCA is a subset)

                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                      PCA Rediscovery ndash Renaming

                                      Statistics Principal Component Analysis (PCA)

                                      Social Sciences Factor Analysis (PCA is a subset)

                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                      Geo-Sciences Empirical Orthogonal Functions (EOF)

                                      An Interesting Historical Note

                                      The 1st () application of PCA to Functional

                                      Data Analysis

                                      An Interesting Historical Note

                                      The 1st () application of PCA to Functional

                                      Data Analysis

                                      Rao (1958)

                                      1st Paper with ldquoCurves as Data Objectsrdquo

                                      viewpoint

                                      Detailed Look at PCA

                                      Three Important (amp Interesting) Viewpoints

                                      1 Mathematics

                                      2 Numerics

                                      3 Statistics

                                      Goal Study Interrelationships

                                      Detailed Look at PCA

                                      Three Important (amp Interesting) Viewpoints

                                      1 Mathematics

                                      2 Numerics

                                      3 Statistics

                                      1st Review Linear Alg and Multivar Prob

                                      Review of Linear Algebra

                                      Vector Space

                                      bull set of ldquovectorsrdquo

                                      bull and ldquoscalarsrdquo (coefficients)

                                      xa

                                      Review of Linear Algebra

                                      Vector Space

                                      bull set of ldquovectorsrdquo

                                      bull and ldquoscalarsrdquo (coefficients)

                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                      ( in space)

                                      xa

                                      i

                                      ii xa

                                      Review of Linear Algebra

                                      Vector Space

                                      bull set of ldquovectorsrdquo

                                      bull and ldquoscalarsrdquo (coefficients)

                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                      ( in space)

                                      eg

                                      ldquo dim Euclidrsquon spacerdquo

                                      xa

                                      i

                                      ii xa

                                      d

                                      d

                                      d xx

                                      x

                                      x

                                      x 1

                                      1

                                      d

                                      Review of Linear Algebra (Cont)

                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                      Review of Linear Algebra (Cont)

                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                      Review of Linear Algebra (Cont)

                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                      Note Planes not Through the Origin

                                      are not Subspaces

                                      (Do not Contain )00 x

                                      Review of Linear Algebra (Cont)

                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                      (all Linear Combos of them =

                                      = Containing Hyperplane

                                      through Origin)

                                      Review of Linear Algebra (Cont)

                                      Basis of Subspace Set of Vectors that

                                      bull Span ie Everything is a Lin Com of them

                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                      Review of Linear Algebra (Cont)

                                      Basis of Subspace Set of Vectors that

                                      bull Span ie Everything is a Lin Com of them

                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                      bull eg ldquoUnit Vector Basisrdquo

                                      bull Since

                                      d

                                      1

                                      0

                                      0

                                      0

                                      1

                                      0

                                      0

                                      0

                                      1

                                      1

                                      0

                                      0

                                      0

                                      1

                                      0

                                      0

                                      0

                                      1

                                      212

                                      1

                                      d

                                      d

                                      xxx

                                      x

                                      x

                                      x

                                      Review of Linear Algebra (Cont)

                                      Basis Matrix of subspace of

                                      Given a basis

                                      create matrix of columns

                                      dnvv 1

                                      nddnd

                                      n

                                      n

                                      vv

                                      vv

                                      vvB

                                      1

                                      111

                                      1

                                      Review of Linear Algebra (Cont)

                                      Then linear combo is a matrix multiplicatrsquon

                                      where

                                      n

                                      iii aBva

                                      1

                                      na

                                      a

                                      a 1

                                      Review of Linear Algebra (Cont)

                                      Then linear combo is a matrix multiplicatrsquon

                                      where

                                      Note Right Multiplication Gives

                                      Linear Combination of Column Vectors

                                      n

                                      iii aBva

                                      1

                                      na

                                      a

                                      a 1

                                      Review of Linear Algebra (Cont)

                                      Then linear combo is a matrix multiplicatrsquon

                                      where

                                      Check sizes

                                      n

                                      iii aBva

                                      1

                                      na

                                      a

                                      a 1

                                      )1()(1 nndd

                                      Review of Linear Algebra (Cont)

                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                      For matrices

                                      Define the Matrix Product

                                      mkk

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      nmm

                                      n

                                      bb

                                      bb

                                      B

                                      1

                                      111

                                      m

                                      iniik

                                      m

                                      iiik

                                      m

                                      inii

                                      m

                                      iii

                                      baba

                                      baba

                                      AB

                                      1

                                      11

                                      11

                                      111

                                      Review of Linear Algebra (Cont)

                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                      For matrices

                                      Define the Matrix Product

                                      (Inner Products of Rows With Columns )

                                      mkk

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      nmm

                                      n

                                      bb

                                      bb

                                      B

                                      1

                                      111

                                      m

                                      iniik

                                      m

                                      iiik

                                      m

                                      inii

                                      m

                                      iii

                                      baba

                                      baba

                                      AB

                                      1

                                      11

                                      11

                                      111

                                      A B

                                      Review of Linear Algebra (Cont)

                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                      For matrices

                                      Define the Matrix Product

                                      (Inner Products of Rows With Columns )

                                      (Composition of Linear Transformations)

                                      mkk

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      nmm

                                      n

                                      bb

                                      bb

                                      B

                                      1

                                      111

                                      m

                                      iniik

                                      m

                                      iiik

                                      m

                                      inii

                                      m

                                      iii

                                      baba

                                      baba

                                      AB

                                      1

                                      11

                                      11

                                      111

                                      A B

                                      Review of Linear Algebra (Cont)

                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                      For matrices

                                      Define the Matrix Product

                                      (Inner Products of Rows With Columns )

                                      (Composition of Linear Transformations)

                                      Often Useful to Check Sizes

                                      mkk

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      nmm

                                      n

                                      bb

                                      bb

                                      B

                                      1

                                      111

                                      m

                                      iniik

                                      m

                                      iiik

                                      m

                                      inii

                                      m

                                      iii

                                      baba

                                      baba

                                      AB

                                      1

                                      11

                                      11

                                      111

                                      nmmknk

                                      A B

                                      Review of Linear Algebra (Cont)

                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                      For matrices

                                      Define the Matrix Product

                                      (Inner Products of Rows With Columns )

                                      (Composition of Linear Transformations)

                                      Often Useful to Check Sizes

                                      mkk

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      nmm

                                      n

                                      bb

                                      bb

                                      B

                                      1

                                      111

                                      m

                                      iniik

                                      m

                                      iiik

                                      m

                                      inii

                                      m

                                      iii

                                      baba

                                      baba

                                      AB

                                      1

                                      11

                                      11

                                      111

                                      nmmknk

                                      A B

                                      Review of Linear Algebra (Cont)

                                      Matrix Trace

                                      bull For a Square Matrix

                                      bull Define

                                      mmm

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      m

                                      iiiaAtr

                                      1)(

                                      Review of Linear Algebra (Cont)

                                      Matrix Trace

                                      bull For a Square Matrix

                                      bull Define

                                      bull Trace Commutes with Matrix Multiplication

                                      mmm

                                      m

                                      aa

                                      aa

                                      A

                                      1

                                      111

                                      m

                                      iiiaAtr

                                      1)(

                                      BAtrABtr

                                      Review of Linear Algebra (Cont)

                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                      bull Number of Elements in a Basis (Unique)

                                      Review of Linear Algebra (Cont)

                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                      bull Number of Elements in a Basis (Unique)

                                      bull (Use Basis Above)

                                      bull eg dim of a line is 1

                                      bull eg dim of a plane is 2

                                      dd dim

                                      Review of Linear Algebra (Cont)

                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                      bull Number of Elements in a Basis (Unique)

                                      bull (Use Basis Above)

                                      bull eg dim of a line is 1

                                      bull eg dim of a plane is 2

                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                      (in Statistical Uses eg ANOVA)

                                      dd dim

                                      Review of Linear Algebra (Cont)

                                      Norm of a Vector

                                      bull in d 21

                                      21

                                      1

                                      2 xxxx td

                                      jj

                                      Review of Linear Algebra (Cont)

                                      Norm of a Vector

                                      bull in

                                      bull Idea length of the vector

                                      d 2121

                                      1

                                      2 xxxx td

                                      jj

                                      Review of Linear Algebra (Cont)

                                      Norm of a Vector

                                      bull in

                                      bull Idea length of the vector

                                      bull Note strange properties for high

                                      eg ldquolength of diagonal of unit cuberdquo =

                                      d 2121

                                      1

                                      2 xxxx td

                                      jj

                                      d

                                      d

                                      Review of Linear Algebra (Cont)

                                      Norm of a Vector (cont)

                                      bull Length Normalized Vector

                                      (has Length 1 thus on Surf of Unit Sphere

                                      amp is a Direction Vector)

                                      x

                                      x

                                      Review of Linear Algebra (Cont)

                                      Norm of a Vector (cont)

                                      bull Length Normalized Vector

                                      (has Length 1 thus on Surf of Unit Sphere

                                      amp is a Direction Vector)

                                      bull Define Distance as

                                      x

                                      x

                                      yxyxyxyxd t

                                      Review of Linear Algebra (Cont)

                                      Inner (Dot Scalar) Product

                                      bull for Vectors and

                                      yxyxyx td

                                      jjj

                                      1

                                      x y

                                      Review of Linear Algebra (Cont)

                                      Inner (Dot Scalar) Product

                                      bull for Vectors and

                                      bull Related to Norm via

                                      yxyxyx td

                                      jjj

                                      1

                                      x y

                                      21 xxx

                                      Review of Linear Algebra (Cont)

                                      Inner (Dot Scalar) Product (cont)

                                      bull measures ldquoangle between and rdquo as

                                      yyxx

                                      yx

                                      yx

                                      yxyxangle

                                      tt

                                      t

                                      11 cos

                                      cos

                                      x y

                                      Review of Linear Algebra (Cont)

                                      Inner (Dot Scalar) Product (cont)

                                      bull measures ldquoangle between and rdquo as

                                      bull key to Orthogonality ie Perpendiculrsquoty

                                      if and only if

                                      yyxx

                                      yx

                                      yx

                                      yxyxangle

                                      tt

                                      t

                                      11 cos

                                      cos

                                      x y

                                      yx 0 yx

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis

                                      bull All Orthogonal to each other

                                      ie for

                                      bull All have Length 1

                                      ie for

                                      nvv 1

                                      1 ii vv

                                      0 ii vv ii

                                      ni 1

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis (cont)

                                      bull Spectral Representation

                                      where

                                      nvv 1

                                      n

                                      iii vax

                                      1

                                      ii vxa

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis (cont)

                                      bull Spectral Representation

                                      where

                                      (Coefficient is Inner Product Cool Notation)

                                      nvv 1

                                      n

                                      iii vax

                                      1

                                      ii vxa

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis (cont)

                                      bull Spectral Representation

                                      where

                                      Check

                                      nvv 1

                                      n

                                      iii vax

                                      1

                                      ii vxa

                                      iii

                                      n

                                      iii

                                      n

                                      iiii avvavvavx

                                      1

                                      1

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis (cont)

                                      bull Spectral Representation

                                      where

                                      Check

                                      bull Matrix Notation where ie

                                      For the Basis Matrix

                                      nvv 1

                                      n

                                      iii vax

                                      1

                                      ii vxa

                                      iii

                                      n

                                      iii

                                      n

                                      iiii avvavvavx

                                      1

                                      1

                                      aBx Bxa tt xBa t

                                      nvvB 1

                                      Review of Linear Algebra (Cont)

                                      Orthonormal Basis (cont)

                                      bull Spectral Representation

                                      where

                                      Check

                                      bull Matrix Notation where ie

                                      is called transform of

                                      (eg Fourier or Wavelet)

                                      nvv 1

                                      n

                                      iii vax

                                      1

                                      ii vxa

                                      iii

                                      n

                                      iii

                                      n

                                      iiii avvavvavx

                                      1

                                      1

                                      aBx Bxa tt xBa t

                                      xa

                                      Review of Linear Algebra (Cont)

                                      Parseval identity for

                                      in subsp genrsquod by o n basis

                                      x

                                      nvv 1

                                      2

                                      1

                                      22

                                      1

                                      2 aavxx

                                      n

                                      ii

                                      n

                                      ii

                                      Review of Linear Algebra (Cont)

                                      Parseval identity for

                                      in subsp genrsquod by o n basis

                                      bull Pythagorean theorem

                                      bull ldquoDecomposition of Energyrdquo

                                      bull ANOVA - sums of squares

                                      x

                                      nvv 1

                                      2

                                      1

                                      22

                                      1

                                      2 aavxx

                                      n

                                      ii

                                      n

                                      ii

                                      Review of Linear Algebra (Cont)

                                      Parseval identity for

                                      in subsp genrsquod by o n basis

                                      bull Pythagorean theorem

                                      bull ldquoDecomposition of Energyrdquo

                                      bull ANOVA - sums of squares

                                      bull Transform has same length as

                                      ie ldquorotation in rdquo

                                      x

                                      nvv 1

                                      2

                                      1

                                      22

                                      1

                                      2 aavxx

                                      n

                                      ii

                                      n

                                      ii

                                      a xd

                                      Projection of a Vector onto a Subspace

                                      bull Idea Member of that is Closest to

                                      (ie ldquoBest Approxrsquonrdquo)

                                      Review of Linear Algebra (Cont)x

                                      xV

                                      V

                                      Projection of a Vector onto a Subspace

                                      bull Idea Member of that is Closest to

                                      (ie ldquoBest Approxrsquonrdquo)

                                      bull Find that Solves

                                      (ldquoLeast Squaresrdquo)

                                      Review of Linear Algebra (Cont)x

                                      xV

                                      V

                                      VxPV vxVv

                                      min

                                      Projection of a Vector onto a Subspace

                                      bull Idea Member of that is Closest to

                                      (ie ldquoBest Approxrsquonrdquo)

                                      bull Find that Solves

                                      (ldquoLeast Squaresrdquo)

                                      bull For Inner Product (Hilbert) Space

                                      Exists and is Unique

                                      Review of Linear Algebra (Cont)x

                                      xV

                                      V

                                      VxPV vxVv

                                      min

                                      xPV

                                      Projection of a Vector onto a Subspace (cont)

                                      bull General Solution in for Basis Matrix

                                      Review of Linear Algebra (Cont)

                                      d VB

                                      xBBBBxP tVV

                                      tVVV

                                      1

                                      Projection of a Vector onto a Subspace (cont)

                                      bull General Solution in for Basis Matrix

                                      bull So Projrsquon Operator is Matrix Multrsquon

                                      (thus projection is another linear operation)

                                      Review of Linear Algebra (Cont)

                                      d VB

                                      xBBBBxP tVV

                                      tVVV

                                      1

                                      tVV

                                      tVVV BBBBP

                                      1

                                      Projection of a Vector onto a Subspace (cont)

                                      bull General Solution in for Basis Matrix

                                      bull So Projrsquon Operator is Matrix Multrsquon

                                      (thus projection is another linear operation)

                                      (note same operation underlies least squares)

                                      Review of Linear Algebra (Cont)

                                      d VB

                                      xBBBBxP tVV

                                      tVVV

                                      1

                                      tVV

                                      tVVV BBBBP

                                      1

                                      Review of Linear Algebra (Cont)

                                      Projection using Orthonormal Basis

                                      bull Basis Matrix is Orthonormal

                                      nnVtV IBB

                                      10

                                      01

                                      1

                                      111

                                      1

                                      1

                                      nnn

                                      n

                                      ntn

                                      t

                                      vvvv

                                      vvvv

                                      vv

                                      v

                                      v

                                      nvv 1

                                      Review of Linear Algebra (Cont)

                                      Projection using Orthonormal Basis

                                      bull Basis Matrix is Orthonormal

                                      bull So =

                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                      (Recall Right Multrsquon)

                                      nnVtV IBB

                                      10

                                      01

                                      1

                                      111

                                      1

                                      1

                                      nnn

                                      n

                                      ntn

                                      t

                                      vvvv

                                      vvvv

                                      vv

                                      v

                                      v

                                      xBBxP tVVV

                                      x V

                                      nvv 1

                                      Review of Linear Algebra (Cont)

                                      Projection using Orthonormal Basis (cont)

                                      bull For Orthogonal Complement

                                      and

                                      V

                                      xPxPx VV 222xPxPx VV

                                      Review of Linear Algebra (Cont)

                                      Projection using Orthonormal Basis (cont)

                                      bull For Orthogonal Complement

                                      and

                                      bull Parseval Inequality

                                      V

                                      xPxPx VV 222xPxPx VV

                                      2

                                      1

                                      22

                                      1

                                      22 aavxxxP

                                      n

                                      ii

                                      n

                                      iiV

                                      Review of Linear Algebra (Cont)

                                      (Real) Unitary Matrices with

                                      bull Orthonormal Basis Matrix

                                      (So All of Above Applies)

                                      ddU IUU t

                                      Review of Linear Algebra (Cont)

                                      (Real) Unitary Matrices with

                                      bull Orthonormal Basis Matrix

                                      (So All of Above Applies)

                                      bull Note Transformrsquon is Distance Preserving

                                      ddU IUU t

                                      yxdyxyxyxUyUxUdn

                                      i ii 2

                                      1

                                      Review of Linear Algebra (Cont)

                                      (Real) Unitary Matrices with

                                      bull Orthonormal Basis Matrix

                                      (So All of Above Applies)

                                      bull Note Transformrsquon is Distance Preserving

                                      bull Lin Trans (Mult by ) is ~ Rotation

                                      bull But also Includes ldquoMirror Imagesrdquo

                                      ddU IUU t

                                      yxdyxyxyxUyUxUdn

                                      i ii 2

                                      1

                                      U

                                      Review of Linear Algebra (Cont)

                                      Singular Value Decomposition (SVD)

                                      For a Matrix

                                      Find

                                      ndX

                                      Review of Linear Algebra (Cont)

                                      Singular Value Decomposition (SVD)

                                      For a Matrix

                                      Find a Diagonal Matrix

                                      with Entries

                                      called Singular Values

                                      ndX

                                      ndS

                                      )min(1 ndss

                                      Review of Linear Algebra (Cont)

                                      Singular Value Decomposition (SVD)

                                      For a Matrix

                                      Find a Diagonal Matrix

                                      with Entries

                                      called Singular Values

                                      And Unitary (Rotation) Matrices

                                      (recall )

                                      ndX

                                      ndS

                                      )min(1 ndss

                                      ddU nnV

                                      IVVUU tt

                                      Review of Linear Algebra (Cont)

                                      Singular Value Decomposition (SVD)

                                      For a Matrix

                                      Find a Diagonal Matrix

                                      with Entries

                                      called Singular Values

                                      And Unitary (Rotation) Matrices

                                      (recall )

                                      So That

                                      ndX

                                      ndS

                                      )min(1 ndss

                                      ddU nnV

                                      IVVUU tt tUSVX

                                      Review of Linear Algebra (Cont)

                                      Intuition behind Singular Value Decomposition

                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                      vVSUvVSUvX tt

                                      Review of Linear Algebra (Cont)

                                      Intuition behind Singular Value Decomposition

                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                      bull First rotate

                                      X

                                      vVSUvVSUvX tt

                                      Review of Linear Algebra (Cont)

                                      Intuition behind Singular Value Decomposition

                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                      bull First rotate

                                      bull Second rescale coordinate axes (by )

                                      X

                                      vVSUvVSUvX tt

                                      is

                                      Review of Linear Algebra (Cont)

                                      Intuition behind Singular Value Decomposition

                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                      bull First rotate

                                      bull Second rescale coordinate axes (by )

                                      bull Third rotate again

                                      X

                                      vVSUvVSUvX tt

                                      is

                                      Review of Linear Algebra (Cont)

                                      Intuition behind Singular Value Decomposition

                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                      bull First rotate

                                      bull Second rescale coordinate axes (by )

                                      bull Third rotate again

                                      bull ie have diagonalized the transformation

                                      X

                                      vVSUvVSUvX tt

                                      is

                                      SVD Compact Representation

                                      Useful Labeling

                                      Singular Values in Increasing Order

                                      Review of Linear Algebra (Cont)

                                      )min(1 dnss

                                      SVD Compact Representation

                                      Useful Labeling

                                      Singular Values in Increasing Order

                                      Note singular values = 0 can be omitted

                                      (Since do ldquo0-Stretchingrdquo)

                                      Review of Linear Algebra (Cont)

                                      )min(1 dnss

                                      Review of Linear Algebra (Cont)

                                      r

                                      SVD Compact Representation

                                      Useful Labeling

                                      Singular Values in Increasing Order

                                      Note singular values = 0 can be omitted

                                      Let = of positive singular values

                                      )min(1 dnss

                                      Review of Linear Algebra (Cont)

                                      r

                                      SVD Compact Representation

                                      Useful Labeling

                                      Singular Values in Increasing Order

                                      Note singular values = 0 can be omitted

                                      Let = of positive singular values

                                      Then

                                      Where are truncations of

                                      trnrrrd VSUX

                                      VSU

                                      )min(1 dnss

                                      Review of Linear Algebra (Cont)

                                      SVD Full Representation

                                      =

                                      Graphics Display Assumes

                                      ndX ddU ndS nn

                                      tV

                                      nd

                                      Review of Linear Algebra (Cont)

                                      SVD Full Representation

                                      =

                                      Full Rank Basis Matrix

                                      ndX ddU ndS nn

                                      tV

                                      Review of Linear Algebra (Cont)

                                      SVD Full Representation

                                      =

                                      Full Rank Basis Matrix

                                      All 0s in Bottom

                                      ndX ddU ndS nn

                                      tV

                                      Review of Linear Algebra (Cont)

                                      SVD Reduced Representation

                                      =

                                      These Columns Get 0ed Out

                                      ndX ddU nnS nn

                                      tV

                                      nnd 0

                                      Review of Linear Algebra (Cont)

                                      SVD Reduced Representation

                                      =ndX ndU nnS nn

                                      tV

                                      Review of Linear Algebra (Cont)

                                      SVD Reduced Representation

                                      =

                                      Also Some of These May be 0

                                      ndX ndU nnS nn

                                      tV

                                      Review of Linear Algebra (Cont)

                                      SVD Compact Representation

                                      = ndX rdU

                                      rrS nrtV

                                      0

                                      Review of Linear Algebra (Cont)

                                      SVD Compact Representation

                                      =

                                      These Get 0ed Out

                                      ndX rdU

                                      rrS nrtV

                                      0

                                      Review of Linear Algebra (Cont)

                                      SVD Compact Representation

                                      = ndX rdU

                                      rrS nrtV

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition

                                      For a (Symmetric) Square Matrix

                                      Find

                                      ddX

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition

                                      For a (Symmetric) Square Matrix

                                      Find a Diagonal Matrix

                                      Called Eigenvalues

                                      Convenient Ordering

                                      ddX

                                      d

                                      D

                                      0

                                      01

                                      n 1

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition

                                      For a (Symmetric) Square Matrix

                                      Find a Diagonal Matrix

                                      And an Orthonormal Matrix

                                      (ie )

                                      ddX

                                      d

                                      D

                                      0

                                      01

                                      ddB

                                      ddtt IBBBB

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition

                                      For a (Symmetric) Square Matrix

                                      Find a Diagonal Matrix

                                      And an Orthonormal Matrix

                                      (ie )

                                      So that ie

                                      ddX

                                      d

                                      D

                                      0

                                      01

                                      ddB

                                      ddtt IBBBB

                                      DBBX tBDBX

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                      (looks similar)

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                      Complex (uses )

                                      VU

                                      1i

                                      Review of Linear Algebra (Cont)

                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                      VU

                                      X

                                      BVU

                                      1i

                                      • Participant Presentations
                                      • PCA to find clusters
                                      • Statistical Smoothing
                                      • Kernel Density Estimation
                                      • Scatterplot Smoothing
                                      • Statistical Smoothing (2)
                                      • SiZer Background
                                      • SiZer Background (2)
                                      • SiZer Background (3)
                                      • SiZer Background (4)
                                      • SiZer Background (5)
                                      • SiZer Overview
                                      • PCA to find clusters (2)
                                      • PCA to find clusters (3)
                                      • PCA to find clusters (4)
                                      • PCA to find clusters (5)
                                      • PCA to find clusters (6)
                                      • PCA to find clusters (7)
                                      • Recall Yeast Cell Cycle Data
                                      • Yeast Cell Cycle Data FDA View
                                      • Yeast Cell Cycle Data FDA View (2)
                                      • Yeast Cell Cycle Data FDA View (3)
                                      • Yeast Cell Cycles Freq 2 Proj
                                      • Frequency 2 Analysis
                                      • Frequency 2 Analysis (2)
                                      • Frequency 2 Analysis (3)
                                      • Yeast Cell Cycle
                                      • SiZer Study of Distrsquon of Angles
                                      • Reclassification of Major Genes
                                      • Compare to Previous Classifrsquon
                                      • New Subpopulation View
                                      • New Subpopulation View (2)
                                      • Detailed Look at PCA
                                      • PCA Rediscovery ndash Renaming
                                      • PCA Rediscovery ndash Renaming (2)
                                      • PCA Rediscovery ndash Renaming (3)
                                      • PCA Rediscovery ndash Renaming (4)
                                      • PCA Rediscovery ndash Renaming (5)
                                      • An Interesting Historical Note
                                      • An Interesting Historical Note (2)
                                      • Detailed Look at PCA (2)
                                      • Detailed Look at PCA (3)
                                      • Review of Linear Algebra
                                      • Review of Linear Algebra (2)
                                      • Review of Linear Algebra (3)
                                      • Review of Linear Algebra (Cont)
                                      • Review of Linear Algebra (Cont) (2)
                                      • Review of Linear Algebra (Cont) (3)
                                      • Review of Linear Algebra (Cont) (4)
                                      • Review of Linear Algebra (Cont) (5)
                                      • Review of Linear Algebra (Cont) (6)
                                      • Review of Linear Algebra (Cont) (7)
                                      • Review of Linear Algebra (Cont) (8)
                                      • Review of Linear Algebra (Cont) (9)
                                      • Review of Linear Algebra (Cont) (10)
                                      • Review of Linear Algebra (Cont) (11)
                                      • Review of Linear Algebra (Cont) (12)
                                      • Review of Linear Algebra (Cont) (13)
                                      • Review of Linear Algebra (Cont) (14)
                                      • Review of Linear Algebra (Cont) (15)
                                      • Review of Linear Algebra (Cont) (16)
                                      • Review of Linear Algebra (Cont) (17)
                                      • Review of Linear Algebra (Cont) (18)
                                      • Review of Linear Algebra (Cont) (19)
                                      • Review of Linear Algebra (Cont) (20)
                                      • Review of Linear Algebra (Cont) (21)
                                      • Review of Linear Algebra (Cont) (22)
                                      • Review of Linear Algebra (Cont) (23)
                                      • Review of Linear Algebra (Cont) (24)
                                      • Review of Linear Algebra (Cont) (25)
                                      • Review of Linear Algebra (Cont) (26)
                                      • Review of Linear Algebra (Cont) (27)
                                      • Review of Linear Algebra (Cont) (28)
                                      • Review of Linear Algebra (Cont) (29)
                                      • Review of Linear Algebra (Cont) (30)
                                      • Review of Linear Algebra (Cont) (31)
                                      • Review of Linear Algebra (Cont) (32)
                                      • Review of Linear Algebra (Cont) (33)
                                      • Review of Linear Algebra (Cont) (34)
                                      • Review of Linear Algebra (Cont) (35)
                                      • Review of Linear Algebra (Cont) (36)
                                      • Review of Linear Algebra (Cont) (37)
                                      • Review of Linear Algebra (Cont) (38)
                                      • Review of Linear Algebra (Cont) (39)
                                      • Review of Linear Algebra (Cont) (40)
                                      • Review of Linear Algebra (Cont) (41)
                                      • Review of Linear Algebra (Cont) (42)
                                      • Review of Linear Algebra (Cont) (43)
                                      • Review of Linear Algebra (Cont) (44)
                                      • Review of Linear Algebra (Cont) (45)
                                      • Review of Linear Algebra (Cont) (46)
                                      • Review of Linear Algebra (Cont) (47)
                                      • Review of Linear Algebra (Cont) (48)
                                      • Review of Linear Algebra (Cont) (49)
                                      • Review of Linear Algebra (Cont) (50)
                                      • Review of Linear Algebra (Cont) (51)
                                      • Review of Linear Algebra (Cont) (52)
                                      • Review of Linear Algebra (Cont) (53)
                                      • Review of Linear Algebra (Cont) (54)
                                      • Review of Linear Algebra (Cont) (55)
                                      • Review of Linear Algebra (Cont) (56)
                                      • Review of Linear Algebra (Cont) (57)
                                      • Review of Linear Algebra (Cont) (58)
                                      • Review of Linear Algebra (Cont) (59)
                                      • Review of Linear Algebra (Cont) (60)
                                      • Review of Linear Algebra (Cont) (61)
                                      • Review of Linear Algebra (Cont) (62)
                                      • Review of Linear Algebra (Cont) (63)
                                      • Review of Linear Algebra (Cont) (64)
                                      • Review of Linear Algebra (Cont) (65)
                                      • Review of Linear Algebra (Cont) (66)
                                      • Review of Linear Algebra (Cont) (67)
                                      • Review of Linear Algebra (Cont) (68)
                                      • Review of Linear Algebra (Cont) (69)
                                      • Review of Linear Algebra (Cont) (70)
                                      • Review of Linear Algebra (Cont) (71)
                                      • Review of Linear Algebra (Cont) (72)
                                      • Review of Linear Algebra (Cont) (73)
                                      • Review of Linear Algebra (Cont) (74)
                                      • Review of Linear Algebra (Cont) (75)
                                      • Review of Linear Algebra (Cont) (76)
                                      • Review of Linear Algebra (Cont) (77)
                                      • Review of Linear Algebra (Cont) (78)
                                      • Review of Linear Algebra (Cont) (79)
                                      • Review of Linear Algebra (Cont) (80)
                                      • Review of Linear Algebra (Cont) (81)

                                        Yeast Cell Cycle Data FDA View

                                        Central questionWhich genes are ldquoperiodicrdquo over 2 cell cycles

                                        Yeast Cell Cycle Data FDA View

                                        Periodic genes

                                        Naiumlve

                                        approach

                                        Simple PCA

                                        Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                        ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                        ldquovariationrdquobull PCA finds ldquodirections of maximal

                                        variationrdquobull Often but not always same as

                                        ldquointeresting directionsrdquobull Here need better approach to study

                                        periodicities

                                        Yeast Cell Cycles Freq 2 Proj

                                        PCA on

                                        Freq 2

                                        Periodic

                                        Component

                                        Of Data

                                        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                        Approach from Zhao Marron amp Wells (2004)

                                        Frequency 2 Analysis

                                        Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                        cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                        classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                        notable differencesbull Now try to improve ldquophase classificationrdquo

                                        Yeast Cell CycleRevisit ldquophase classificationrdquo

                                        approachbull Use outer 200 genes

                                        (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                        (finds significant bumps etc in histogram)

                                        bull Carefully redrew boundariesbull Check by studying kde angles

                                        SiZer Study of Distrsquon of Angles

                                        Reclassification of Major Genes

                                        Compare to Previous Classifrsquon

                                        New Subpopulation View

                                        New Subpopulation View

                                        NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                        Detailed Look at PCA

                                        Now Study ldquoFolklorerdquo More Carefully

                                        bull BackGround

                                        bull History

                                        bull Underpinnings

                                        (Mathematical amp Computational)

                                        Good Overall Reference Jolliffe (2002)

                                        PCA Rediscovery ndash Renaming

                                        Statistics Principal Component Analysis (PCA)

                                        PCA Rediscovery ndash Renaming

                                        Statistics Principal Component Analysis (PCA)

                                        Social Sciences Factor Analysis (PCA is a subset)

                                        PCA Rediscovery ndash Renaming

                                        Statistics Principal Component Analysis (PCA)

                                        Social Sciences Factor Analysis (PCA is a subset)

                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                        PCA Rediscovery ndash Renaming

                                        Statistics Principal Component Analysis (PCA)

                                        Social Sciences Factor Analysis (PCA is a subset)

                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                        PCA Rediscovery ndash Renaming

                                        Statistics Principal Component Analysis (PCA)

                                        Social Sciences Factor Analysis (PCA is a subset)

                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                        Geo-Sciences Empirical Orthogonal Functions (EOF)

                                        An Interesting Historical Note

                                        The 1st () application of PCA to Functional

                                        Data Analysis

                                        An Interesting Historical Note

                                        The 1st () application of PCA to Functional

                                        Data Analysis

                                        Rao (1958)

                                        1st Paper with ldquoCurves as Data Objectsrdquo

                                        viewpoint

                                        Detailed Look at PCA

                                        Three Important (amp Interesting) Viewpoints

                                        1 Mathematics

                                        2 Numerics

                                        3 Statistics

                                        Goal Study Interrelationships

                                        Detailed Look at PCA

                                        Three Important (amp Interesting) Viewpoints

                                        1 Mathematics

                                        2 Numerics

                                        3 Statistics

                                        1st Review Linear Alg and Multivar Prob

                                        Review of Linear Algebra

                                        Vector Space

                                        bull set of ldquovectorsrdquo

                                        bull and ldquoscalarsrdquo (coefficients)

                                        xa

                                        Review of Linear Algebra

                                        Vector Space

                                        bull set of ldquovectorsrdquo

                                        bull and ldquoscalarsrdquo (coefficients)

                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                        ( in space)

                                        xa

                                        i

                                        ii xa

                                        Review of Linear Algebra

                                        Vector Space

                                        bull set of ldquovectorsrdquo

                                        bull and ldquoscalarsrdquo (coefficients)

                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                        ( in space)

                                        eg

                                        ldquo dim Euclidrsquon spacerdquo

                                        xa

                                        i

                                        ii xa

                                        d

                                        d

                                        d xx

                                        x

                                        x

                                        x 1

                                        1

                                        d

                                        Review of Linear Algebra (Cont)

                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                        Review of Linear Algebra (Cont)

                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                        Review of Linear Algebra (Cont)

                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                        Note Planes not Through the Origin

                                        are not Subspaces

                                        (Do not Contain )00 x

                                        Review of Linear Algebra (Cont)

                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                        (all Linear Combos of them =

                                        = Containing Hyperplane

                                        through Origin)

                                        Review of Linear Algebra (Cont)

                                        Basis of Subspace Set of Vectors that

                                        bull Span ie Everything is a Lin Com of them

                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                        Review of Linear Algebra (Cont)

                                        Basis of Subspace Set of Vectors that

                                        bull Span ie Everything is a Lin Com of them

                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                        bull eg ldquoUnit Vector Basisrdquo

                                        bull Since

                                        d

                                        1

                                        0

                                        0

                                        0

                                        1

                                        0

                                        0

                                        0

                                        1

                                        1

                                        0

                                        0

                                        0

                                        1

                                        0

                                        0

                                        0

                                        1

                                        212

                                        1

                                        d

                                        d

                                        xxx

                                        x

                                        x

                                        x

                                        Review of Linear Algebra (Cont)

                                        Basis Matrix of subspace of

                                        Given a basis

                                        create matrix of columns

                                        dnvv 1

                                        nddnd

                                        n

                                        n

                                        vv

                                        vv

                                        vvB

                                        1

                                        111

                                        1

                                        Review of Linear Algebra (Cont)

                                        Then linear combo is a matrix multiplicatrsquon

                                        where

                                        n

                                        iii aBva

                                        1

                                        na

                                        a

                                        a 1

                                        Review of Linear Algebra (Cont)

                                        Then linear combo is a matrix multiplicatrsquon

                                        where

                                        Note Right Multiplication Gives

                                        Linear Combination of Column Vectors

                                        n

                                        iii aBva

                                        1

                                        na

                                        a

                                        a 1

                                        Review of Linear Algebra (Cont)

                                        Then linear combo is a matrix multiplicatrsquon

                                        where

                                        Check sizes

                                        n

                                        iii aBva

                                        1

                                        na

                                        a

                                        a 1

                                        )1()(1 nndd

                                        Review of Linear Algebra (Cont)

                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                        For matrices

                                        Define the Matrix Product

                                        mkk

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        nmm

                                        n

                                        bb

                                        bb

                                        B

                                        1

                                        111

                                        m

                                        iniik

                                        m

                                        iiik

                                        m

                                        inii

                                        m

                                        iii

                                        baba

                                        baba

                                        AB

                                        1

                                        11

                                        11

                                        111

                                        Review of Linear Algebra (Cont)

                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                        For matrices

                                        Define the Matrix Product

                                        (Inner Products of Rows With Columns )

                                        mkk

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        nmm

                                        n

                                        bb

                                        bb

                                        B

                                        1

                                        111

                                        m

                                        iniik

                                        m

                                        iiik

                                        m

                                        inii

                                        m

                                        iii

                                        baba

                                        baba

                                        AB

                                        1

                                        11

                                        11

                                        111

                                        A B

                                        Review of Linear Algebra (Cont)

                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                        For matrices

                                        Define the Matrix Product

                                        (Inner Products of Rows With Columns )

                                        (Composition of Linear Transformations)

                                        mkk

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        nmm

                                        n

                                        bb

                                        bb

                                        B

                                        1

                                        111

                                        m

                                        iniik

                                        m

                                        iiik

                                        m

                                        inii

                                        m

                                        iii

                                        baba

                                        baba

                                        AB

                                        1

                                        11

                                        11

                                        111

                                        A B

                                        Review of Linear Algebra (Cont)

                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                        For matrices

                                        Define the Matrix Product

                                        (Inner Products of Rows With Columns )

                                        (Composition of Linear Transformations)

                                        Often Useful to Check Sizes

                                        mkk

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        nmm

                                        n

                                        bb

                                        bb

                                        B

                                        1

                                        111

                                        m

                                        iniik

                                        m

                                        iiik

                                        m

                                        inii

                                        m

                                        iii

                                        baba

                                        baba

                                        AB

                                        1

                                        11

                                        11

                                        111

                                        nmmknk

                                        A B

                                        Review of Linear Algebra (Cont)

                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                        For matrices

                                        Define the Matrix Product

                                        (Inner Products of Rows With Columns )

                                        (Composition of Linear Transformations)

                                        Often Useful to Check Sizes

                                        mkk

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        nmm

                                        n

                                        bb

                                        bb

                                        B

                                        1

                                        111

                                        m

                                        iniik

                                        m

                                        iiik

                                        m

                                        inii

                                        m

                                        iii

                                        baba

                                        baba

                                        AB

                                        1

                                        11

                                        11

                                        111

                                        nmmknk

                                        A B

                                        Review of Linear Algebra (Cont)

                                        Matrix Trace

                                        bull For a Square Matrix

                                        bull Define

                                        mmm

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        m

                                        iiiaAtr

                                        1)(

                                        Review of Linear Algebra (Cont)

                                        Matrix Trace

                                        bull For a Square Matrix

                                        bull Define

                                        bull Trace Commutes with Matrix Multiplication

                                        mmm

                                        m

                                        aa

                                        aa

                                        A

                                        1

                                        111

                                        m

                                        iiiaAtr

                                        1)(

                                        BAtrABtr

                                        Review of Linear Algebra (Cont)

                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                        bull Number of Elements in a Basis (Unique)

                                        Review of Linear Algebra (Cont)

                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                        bull Number of Elements in a Basis (Unique)

                                        bull (Use Basis Above)

                                        bull eg dim of a line is 1

                                        bull eg dim of a plane is 2

                                        dd dim

                                        Review of Linear Algebra (Cont)

                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                        bull Number of Elements in a Basis (Unique)

                                        bull (Use Basis Above)

                                        bull eg dim of a line is 1

                                        bull eg dim of a plane is 2

                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                        (in Statistical Uses eg ANOVA)

                                        dd dim

                                        Review of Linear Algebra (Cont)

                                        Norm of a Vector

                                        bull in d 21

                                        21

                                        1

                                        2 xxxx td

                                        jj

                                        Review of Linear Algebra (Cont)

                                        Norm of a Vector

                                        bull in

                                        bull Idea length of the vector

                                        d 2121

                                        1

                                        2 xxxx td

                                        jj

                                        Review of Linear Algebra (Cont)

                                        Norm of a Vector

                                        bull in

                                        bull Idea length of the vector

                                        bull Note strange properties for high

                                        eg ldquolength of diagonal of unit cuberdquo =

                                        d 2121

                                        1

                                        2 xxxx td

                                        jj

                                        d

                                        d

                                        Review of Linear Algebra (Cont)

                                        Norm of a Vector (cont)

                                        bull Length Normalized Vector

                                        (has Length 1 thus on Surf of Unit Sphere

                                        amp is a Direction Vector)

                                        x

                                        x

                                        Review of Linear Algebra (Cont)

                                        Norm of a Vector (cont)

                                        bull Length Normalized Vector

                                        (has Length 1 thus on Surf of Unit Sphere

                                        amp is a Direction Vector)

                                        bull Define Distance as

                                        x

                                        x

                                        yxyxyxyxd t

                                        Review of Linear Algebra (Cont)

                                        Inner (Dot Scalar) Product

                                        bull for Vectors and

                                        yxyxyx td

                                        jjj

                                        1

                                        x y

                                        Review of Linear Algebra (Cont)

                                        Inner (Dot Scalar) Product

                                        bull for Vectors and

                                        bull Related to Norm via

                                        yxyxyx td

                                        jjj

                                        1

                                        x y

                                        21 xxx

                                        Review of Linear Algebra (Cont)

                                        Inner (Dot Scalar) Product (cont)

                                        bull measures ldquoangle between and rdquo as

                                        yyxx

                                        yx

                                        yx

                                        yxyxangle

                                        tt

                                        t

                                        11 cos

                                        cos

                                        x y

                                        Review of Linear Algebra (Cont)

                                        Inner (Dot Scalar) Product (cont)

                                        bull measures ldquoangle between and rdquo as

                                        bull key to Orthogonality ie Perpendiculrsquoty

                                        if and only if

                                        yyxx

                                        yx

                                        yx

                                        yxyxangle

                                        tt

                                        t

                                        11 cos

                                        cos

                                        x y

                                        yx 0 yx

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis

                                        bull All Orthogonal to each other

                                        ie for

                                        bull All have Length 1

                                        ie for

                                        nvv 1

                                        1 ii vv

                                        0 ii vv ii

                                        ni 1

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis (cont)

                                        bull Spectral Representation

                                        where

                                        nvv 1

                                        n

                                        iii vax

                                        1

                                        ii vxa

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis (cont)

                                        bull Spectral Representation

                                        where

                                        (Coefficient is Inner Product Cool Notation)

                                        nvv 1

                                        n

                                        iii vax

                                        1

                                        ii vxa

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis (cont)

                                        bull Spectral Representation

                                        where

                                        Check

                                        nvv 1

                                        n

                                        iii vax

                                        1

                                        ii vxa

                                        iii

                                        n

                                        iii

                                        n

                                        iiii avvavvavx

                                        1

                                        1

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis (cont)

                                        bull Spectral Representation

                                        where

                                        Check

                                        bull Matrix Notation where ie

                                        For the Basis Matrix

                                        nvv 1

                                        n

                                        iii vax

                                        1

                                        ii vxa

                                        iii

                                        n

                                        iii

                                        n

                                        iiii avvavvavx

                                        1

                                        1

                                        aBx Bxa tt xBa t

                                        nvvB 1

                                        Review of Linear Algebra (Cont)

                                        Orthonormal Basis (cont)

                                        bull Spectral Representation

                                        where

                                        Check

                                        bull Matrix Notation where ie

                                        is called transform of

                                        (eg Fourier or Wavelet)

                                        nvv 1

                                        n

                                        iii vax

                                        1

                                        ii vxa

                                        iii

                                        n

                                        iii

                                        n

                                        iiii avvavvavx

                                        1

                                        1

                                        aBx Bxa tt xBa t

                                        xa

                                        Review of Linear Algebra (Cont)

                                        Parseval identity for

                                        in subsp genrsquod by o n basis

                                        x

                                        nvv 1

                                        2

                                        1

                                        22

                                        1

                                        2 aavxx

                                        n

                                        ii

                                        n

                                        ii

                                        Review of Linear Algebra (Cont)

                                        Parseval identity for

                                        in subsp genrsquod by o n basis

                                        bull Pythagorean theorem

                                        bull ldquoDecomposition of Energyrdquo

                                        bull ANOVA - sums of squares

                                        x

                                        nvv 1

                                        2

                                        1

                                        22

                                        1

                                        2 aavxx

                                        n

                                        ii

                                        n

                                        ii

                                        Review of Linear Algebra (Cont)

                                        Parseval identity for

                                        in subsp genrsquod by o n basis

                                        bull Pythagorean theorem

                                        bull ldquoDecomposition of Energyrdquo

                                        bull ANOVA - sums of squares

                                        bull Transform has same length as

                                        ie ldquorotation in rdquo

                                        x

                                        nvv 1

                                        2

                                        1

                                        22

                                        1

                                        2 aavxx

                                        n

                                        ii

                                        n

                                        ii

                                        a xd

                                        Projection of a Vector onto a Subspace

                                        bull Idea Member of that is Closest to

                                        (ie ldquoBest Approxrsquonrdquo)

                                        Review of Linear Algebra (Cont)x

                                        xV

                                        V

                                        Projection of a Vector onto a Subspace

                                        bull Idea Member of that is Closest to

                                        (ie ldquoBest Approxrsquonrdquo)

                                        bull Find that Solves

                                        (ldquoLeast Squaresrdquo)

                                        Review of Linear Algebra (Cont)x

                                        xV

                                        V

                                        VxPV vxVv

                                        min

                                        Projection of a Vector onto a Subspace

                                        bull Idea Member of that is Closest to

                                        (ie ldquoBest Approxrsquonrdquo)

                                        bull Find that Solves

                                        (ldquoLeast Squaresrdquo)

                                        bull For Inner Product (Hilbert) Space

                                        Exists and is Unique

                                        Review of Linear Algebra (Cont)x

                                        xV

                                        V

                                        VxPV vxVv

                                        min

                                        xPV

                                        Projection of a Vector onto a Subspace (cont)

                                        bull General Solution in for Basis Matrix

                                        Review of Linear Algebra (Cont)

                                        d VB

                                        xBBBBxP tVV

                                        tVVV

                                        1

                                        Projection of a Vector onto a Subspace (cont)

                                        bull General Solution in for Basis Matrix

                                        bull So Projrsquon Operator is Matrix Multrsquon

                                        (thus projection is another linear operation)

                                        Review of Linear Algebra (Cont)

                                        d VB

                                        xBBBBxP tVV

                                        tVVV

                                        1

                                        tVV

                                        tVVV BBBBP

                                        1

                                        Projection of a Vector onto a Subspace (cont)

                                        bull General Solution in for Basis Matrix

                                        bull So Projrsquon Operator is Matrix Multrsquon

                                        (thus projection is another linear operation)

                                        (note same operation underlies least squares)

                                        Review of Linear Algebra (Cont)

                                        d VB

                                        xBBBBxP tVV

                                        tVVV

                                        1

                                        tVV

                                        tVVV BBBBP

                                        1

                                        Review of Linear Algebra (Cont)

                                        Projection using Orthonormal Basis

                                        bull Basis Matrix is Orthonormal

                                        nnVtV IBB

                                        10

                                        01

                                        1

                                        111

                                        1

                                        1

                                        nnn

                                        n

                                        ntn

                                        t

                                        vvvv

                                        vvvv

                                        vv

                                        v

                                        v

                                        nvv 1

                                        Review of Linear Algebra (Cont)

                                        Projection using Orthonormal Basis

                                        bull Basis Matrix is Orthonormal

                                        bull So =

                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                        (Recall Right Multrsquon)

                                        nnVtV IBB

                                        10

                                        01

                                        1

                                        111

                                        1

                                        1

                                        nnn

                                        n

                                        ntn

                                        t

                                        vvvv

                                        vvvv

                                        vv

                                        v

                                        v

                                        xBBxP tVVV

                                        x V

                                        nvv 1

                                        Review of Linear Algebra (Cont)

                                        Projection using Orthonormal Basis (cont)

                                        bull For Orthogonal Complement

                                        and

                                        V

                                        xPxPx VV 222xPxPx VV

                                        Review of Linear Algebra (Cont)

                                        Projection using Orthonormal Basis (cont)

                                        bull For Orthogonal Complement

                                        and

                                        bull Parseval Inequality

                                        V

                                        xPxPx VV 222xPxPx VV

                                        2

                                        1

                                        22

                                        1

                                        22 aavxxxP

                                        n

                                        ii

                                        n

                                        iiV

                                        Review of Linear Algebra (Cont)

                                        (Real) Unitary Matrices with

                                        bull Orthonormal Basis Matrix

                                        (So All of Above Applies)

                                        ddU IUU t

                                        Review of Linear Algebra (Cont)

                                        (Real) Unitary Matrices with

                                        bull Orthonormal Basis Matrix

                                        (So All of Above Applies)

                                        bull Note Transformrsquon is Distance Preserving

                                        ddU IUU t

                                        yxdyxyxyxUyUxUdn

                                        i ii 2

                                        1

                                        Review of Linear Algebra (Cont)

                                        (Real) Unitary Matrices with

                                        bull Orthonormal Basis Matrix

                                        (So All of Above Applies)

                                        bull Note Transformrsquon is Distance Preserving

                                        bull Lin Trans (Mult by ) is ~ Rotation

                                        bull But also Includes ldquoMirror Imagesrdquo

                                        ddU IUU t

                                        yxdyxyxyxUyUxUdn

                                        i ii 2

                                        1

                                        U

                                        Review of Linear Algebra (Cont)

                                        Singular Value Decomposition (SVD)

                                        For a Matrix

                                        Find

                                        ndX

                                        Review of Linear Algebra (Cont)

                                        Singular Value Decomposition (SVD)

                                        For a Matrix

                                        Find a Diagonal Matrix

                                        with Entries

                                        called Singular Values

                                        ndX

                                        ndS

                                        )min(1 ndss

                                        Review of Linear Algebra (Cont)

                                        Singular Value Decomposition (SVD)

                                        For a Matrix

                                        Find a Diagonal Matrix

                                        with Entries

                                        called Singular Values

                                        And Unitary (Rotation) Matrices

                                        (recall )

                                        ndX

                                        ndS

                                        )min(1 ndss

                                        ddU nnV

                                        IVVUU tt

                                        Review of Linear Algebra (Cont)

                                        Singular Value Decomposition (SVD)

                                        For a Matrix

                                        Find a Diagonal Matrix

                                        with Entries

                                        called Singular Values

                                        And Unitary (Rotation) Matrices

                                        (recall )

                                        So That

                                        ndX

                                        ndS

                                        )min(1 ndss

                                        ddU nnV

                                        IVVUU tt tUSVX

                                        Review of Linear Algebra (Cont)

                                        Intuition behind Singular Value Decomposition

                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                        vVSUvVSUvX tt

                                        Review of Linear Algebra (Cont)

                                        Intuition behind Singular Value Decomposition

                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                        bull First rotate

                                        X

                                        vVSUvVSUvX tt

                                        Review of Linear Algebra (Cont)

                                        Intuition behind Singular Value Decomposition

                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                        bull First rotate

                                        bull Second rescale coordinate axes (by )

                                        X

                                        vVSUvVSUvX tt

                                        is

                                        Review of Linear Algebra (Cont)

                                        Intuition behind Singular Value Decomposition

                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                        bull First rotate

                                        bull Second rescale coordinate axes (by )

                                        bull Third rotate again

                                        X

                                        vVSUvVSUvX tt

                                        is

                                        Review of Linear Algebra (Cont)

                                        Intuition behind Singular Value Decomposition

                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                        bull First rotate

                                        bull Second rescale coordinate axes (by )

                                        bull Third rotate again

                                        bull ie have diagonalized the transformation

                                        X

                                        vVSUvVSUvX tt

                                        is

                                        SVD Compact Representation

                                        Useful Labeling

                                        Singular Values in Increasing Order

                                        Review of Linear Algebra (Cont)

                                        )min(1 dnss

                                        SVD Compact Representation

                                        Useful Labeling

                                        Singular Values in Increasing Order

                                        Note singular values = 0 can be omitted

                                        (Since do ldquo0-Stretchingrdquo)

                                        Review of Linear Algebra (Cont)

                                        )min(1 dnss

                                        Review of Linear Algebra (Cont)

                                        r

                                        SVD Compact Representation

                                        Useful Labeling

                                        Singular Values in Increasing Order

                                        Note singular values = 0 can be omitted

                                        Let = of positive singular values

                                        )min(1 dnss

                                        Review of Linear Algebra (Cont)

                                        r

                                        SVD Compact Representation

                                        Useful Labeling

                                        Singular Values in Increasing Order

                                        Note singular values = 0 can be omitted

                                        Let = of positive singular values

                                        Then

                                        Where are truncations of

                                        trnrrrd VSUX

                                        VSU

                                        )min(1 dnss

                                        Review of Linear Algebra (Cont)

                                        SVD Full Representation

                                        =

                                        Graphics Display Assumes

                                        ndX ddU ndS nn

                                        tV

                                        nd

                                        Review of Linear Algebra (Cont)

                                        SVD Full Representation

                                        =

                                        Full Rank Basis Matrix

                                        ndX ddU ndS nn

                                        tV

                                        Review of Linear Algebra (Cont)

                                        SVD Full Representation

                                        =

                                        Full Rank Basis Matrix

                                        All 0s in Bottom

                                        ndX ddU ndS nn

                                        tV

                                        Review of Linear Algebra (Cont)

                                        SVD Reduced Representation

                                        =

                                        These Columns Get 0ed Out

                                        ndX ddU nnS nn

                                        tV

                                        nnd 0

                                        Review of Linear Algebra (Cont)

                                        SVD Reduced Representation

                                        =ndX ndU nnS nn

                                        tV

                                        Review of Linear Algebra (Cont)

                                        SVD Reduced Representation

                                        =

                                        Also Some of These May be 0

                                        ndX ndU nnS nn

                                        tV

                                        Review of Linear Algebra (Cont)

                                        SVD Compact Representation

                                        = ndX rdU

                                        rrS nrtV

                                        0

                                        Review of Linear Algebra (Cont)

                                        SVD Compact Representation

                                        =

                                        These Get 0ed Out

                                        ndX rdU

                                        rrS nrtV

                                        0

                                        Review of Linear Algebra (Cont)

                                        SVD Compact Representation

                                        = ndX rdU

                                        rrS nrtV

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition

                                        For a (Symmetric) Square Matrix

                                        Find

                                        ddX

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition

                                        For a (Symmetric) Square Matrix

                                        Find a Diagonal Matrix

                                        Called Eigenvalues

                                        Convenient Ordering

                                        ddX

                                        d

                                        D

                                        0

                                        01

                                        n 1

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition

                                        For a (Symmetric) Square Matrix

                                        Find a Diagonal Matrix

                                        And an Orthonormal Matrix

                                        (ie )

                                        ddX

                                        d

                                        D

                                        0

                                        01

                                        ddB

                                        ddtt IBBBB

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition

                                        For a (Symmetric) Square Matrix

                                        Find a Diagonal Matrix

                                        And an Orthonormal Matrix

                                        (ie )

                                        So that ie

                                        ddX

                                        d

                                        D

                                        0

                                        01

                                        ddB

                                        ddtt IBBBB

                                        DBBX tBDBX

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                        (looks similar)

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                        Complex (uses )

                                        VU

                                        1i

                                        Review of Linear Algebra (Cont)

                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                        VU

                                        X

                                        BVU

                                        1i

                                        • Participant Presentations
                                        • PCA to find clusters
                                        • Statistical Smoothing
                                        • Kernel Density Estimation
                                        • Scatterplot Smoothing
                                        • Statistical Smoothing (2)
                                        • SiZer Background
                                        • SiZer Background (2)
                                        • SiZer Background (3)
                                        • SiZer Background (4)
                                        • SiZer Background (5)
                                        • SiZer Overview
                                        • PCA to find clusters (2)
                                        • PCA to find clusters (3)
                                        • PCA to find clusters (4)
                                        • PCA to find clusters (5)
                                        • PCA to find clusters (6)
                                        • PCA to find clusters (7)
                                        • Recall Yeast Cell Cycle Data
                                        • Yeast Cell Cycle Data FDA View
                                        • Yeast Cell Cycle Data FDA View (2)
                                        • Yeast Cell Cycle Data FDA View (3)
                                        • Yeast Cell Cycles Freq 2 Proj
                                        • Frequency 2 Analysis
                                        • Frequency 2 Analysis (2)
                                        • Frequency 2 Analysis (3)
                                        • Yeast Cell Cycle
                                        • SiZer Study of Distrsquon of Angles
                                        • Reclassification of Major Genes
                                        • Compare to Previous Classifrsquon
                                        • New Subpopulation View
                                        • New Subpopulation View (2)
                                        • Detailed Look at PCA
                                        • PCA Rediscovery ndash Renaming
                                        • PCA Rediscovery ndash Renaming (2)
                                        • PCA Rediscovery ndash Renaming (3)
                                        • PCA Rediscovery ndash Renaming (4)
                                        • PCA Rediscovery ndash Renaming (5)
                                        • An Interesting Historical Note
                                        • An Interesting Historical Note (2)
                                        • Detailed Look at PCA (2)
                                        • Detailed Look at PCA (3)
                                        • Review of Linear Algebra
                                        • Review of Linear Algebra (2)
                                        • Review of Linear Algebra (3)
                                        • Review of Linear Algebra (Cont)
                                        • Review of Linear Algebra (Cont) (2)
                                        • Review of Linear Algebra (Cont) (3)
                                        • Review of Linear Algebra (Cont) (4)
                                        • Review of Linear Algebra (Cont) (5)
                                        • Review of Linear Algebra (Cont) (6)
                                        • Review of Linear Algebra (Cont) (7)
                                        • Review of Linear Algebra (Cont) (8)
                                        • Review of Linear Algebra (Cont) (9)
                                        • Review of Linear Algebra (Cont) (10)
                                        • Review of Linear Algebra (Cont) (11)
                                        • Review of Linear Algebra (Cont) (12)
                                        • Review of Linear Algebra (Cont) (13)
                                        • Review of Linear Algebra (Cont) (14)
                                        • Review of Linear Algebra (Cont) (15)
                                        • Review of Linear Algebra (Cont) (16)
                                        • Review of Linear Algebra (Cont) (17)
                                        • Review of Linear Algebra (Cont) (18)
                                        • Review of Linear Algebra (Cont) (19)
                                        • Review of Linear Algebra (Cont) (20)
                                        • Review of Linear Algebra (Cont) (21)
                                        • Review of Linear Algebra (Cont) (22)
                                        • Review of Linear Algebra (Cont) (23)
                                        • Review of Linear Algebra (Cont) (24)
                                        • Review of Linear Algebra (Cont) (25)
                                        • Review of Linear Algebra (Cont) (26)
                                        • Review of Linear Algebra (Cont) (27)
                                        • Review of Linear Algebra (Cont) (28)
                                        • Review of Linear Algebra (Cont) (29)
                                        • Review of Linear Algebra (Cont) (30)
                                        • Review of Linear Algebra (Cont) (31)
                                        • Review of Linear Algebra (Cont) (32)
                                        • Review of Linear Algebra (Cont) (33)
                                        • Review of Linear Algebra (Cont) (34)
                                        • Review of Linear Algebra (Cont) (35)
                                        • Review of Linear Algebra (Cont) (36)
                                        • Review of Linear Algebra (Cont) (37)
                                        • Review of Linear Algebra (Cont) (38)
                                        • Review of Linear Algebra (Cont) (39)
                                        • Review of Linear Algebra (Cont) (40)
                                        • Review of Linear Algebra (Cont) (41)
                                        • Review of Linear Algebra (Cont) (42)
                                        • Review of Linear Algebra (Cont) (43)
                                        • Review of Linear Algebra (Cont) (44)
                                        • Review of Linear Algebra (Cont) (45)
                                        • Review of Linear Algebra (Cont) (46)
                                        • Review of Linear Algebra (Cont) (47)
                                        • Review of Linear Algebra (Cont) (48)
                                        • Review of Linear Algebra (Cont) (49)
                                        • Review of Linear Algebra (Cont) (50)
                                        • Review of Linear Algebra (Cont) (51)
                                        • Review of Linear Algebra (Cont) (52)
                                        • Review of Linear Algebra (Cont) (53)
                                        • Review of Linear Algebra (Cont) (54)
                                        • Review of Linear Algebra (Cont) (55)
                                        • Review of Linear Algebra (Cont) (56)
                                        • Review of Linear Algebra (Cont) (57)
                                        • Review of Linear Algebra (Cont) (58)
                                        • Review of Linear Algebra (Cont) (59)
                                        • Review of Linear Algebra (Cont) (60)
                                        • Review of Linear Algebra (Cont) (61)
                                        • Review of Linear Algebra (Cont) (62)
                                        • Review of Linear Algebra (Cont) (63)
                                        • Review of Linear Algebra (Cont) (64)
                                        • Review of Linear Algebra (Cont) (65)
                                        • Review of Linear Algebra (Cont) (66)
                                        • Review of Linear Algebra (Cont) (67)
                                        • Review of Linear Algebra (Cont) (68)
                                        • Review of Linear Algebra (Cont) (69)
                                        • Review of Linear Algebra (Cont) (70)
                                        • Review of Linear Algebra (Cont) (71)
                                        • Review of Linear Algebra (Cont) (72)
                                        • Review of Linear Algebra (Cont) (73)
                                        • Review of Linear Algebra (Cont) (74)
                                        • Review of Linear Algebra (Cont) (75)
                                        • Review of Linear Algebra (Cont) (76)
                                        • Review of Linear Algebra (Cont) (77)
                                        • Review of Linear Algebra (Cont) (78)
                                        • Review of Linear Algebra (Cont) (79)
                                        • Review of Linear Algebra (Cont) (80)
                                        • Review of Linear Algebra (Cont) (81)

                                          Yeast Cell Cycle Data FDA View

                                          Periodic genes

                                          Naiumlve

                                          approach

                                          Simple PCA

                                          Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                          ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                          ldquovariationrdquobull PCA finds ldquodirections of maximal

                                          variationrdquobull Often but not always same as

                                          ldquointeresting directionsrdquobull Here need better approach to study

                                          periodicities

                                          Yeast Cell Cycles Freq 2 Proj

                                          PCA on

                                          Freq 2

                                          Periodic

                                          Component

                                          Of Data

                                          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                          Approach from Zhao Marron amp Wells (2004)

                                          Frequency 2 Analysis

                                          Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                          cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                          classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                          notable differencesbull Now try to improve ldquophase classificationrdquo

                                          Yeast Cell CycleRevisit ldquophase classificationrdquo

                                          approachbull Use outer 200 genes

                                          (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                          (finds significant bumps etc in histogram)

                                          bull Carefully redrew boundariesbull Check by studying kde angles

                                          SiZer Study of Distrsquon of Angles

                                          Reclassification of Major Genes

                                          Compare to Previous Classifrsquon

                                          New Subpopulation View

                                          New Subpopulation View

                                          NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                          Detailed Look at PCA

                                          Now Study ldquoFolklorerdquo More Carefully

                                          bull BackGround

                                          bull History

                                          bull Underpinnings

                                          (Mathematical amp Computational)

                                          Good Overall Reference Jolliffe (2002)

                                          PCA Rediscovery ndash Renaming

                                          Statistics Principal Component Analysis (PCA)

                                          PCA Rediscovery ndash Renaming

                                          Statistics Principal Component Analysis (PCA)

                                          Social Sciences Factor Analysis (PCA is a subset)

                                          PCA Rediscovery ndash Renaming

                                          Statistics Principal Component Analysis (PCA)

                                          Social Sciences Factor Analysis (PCA is a subset)

                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                          PCA Rediscovery ndash Renaming

                                          Statistics Principal Component Analysis (PCA)

                                          Social Sciences Factor Analysis (PCA is a subset)

                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                          PCA Rediscovery ndash Renaming

                                          Statistics Principal Component Analysis (PCA)

                                          Social Sciences Factor Analysis (PCA is a subset)

                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                          Geo-Sciences Empirical Orthogonal Functions (EOF)

                                          An Interesting Historical Note

                                          The 1st () application of PCA to Functional

                                          Data Analysis

                                          An Interesting Historical Note

                                          The 1st () application of PCA to Functional

                                          Data Analysis

                                          Rao (1958)

                                          1st Paper with ldquoCurves as Data Objectsrdquo

                                          viewpoint

                                          Detailed Look at PCA

                                          Three Important (amp Interesting) Viewpoints

                                          1 Mathematics

                                          2 Numerics

                                          3 Statistics

                                          Goal Study Interrelationships

                                          Detailed Look at PCA

                                          Three Important (amp Interesting) Viewpoints

                                          1 Mathematics

                                          2 Numerics

                                          3 Statistics

                                          1st Review Linear Alg and Multivar Prob

                                          Review of Linear Algebra

                                          Vector Space

                                          bull set of ldquovectorsrdquo

                                          bull and ldquoscalarsrdquo (coefficients)

                                          xa

                                          Review of Linear Algebra

                                          Vector Space

                                          bull set of ldquovectorsrdquo

                                          bull and ldquoscalarsrdquo (coefficients)

                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                          ( in space)

                                          xa

                                          i

                                          ii xa

                                          Review of Linear Algebra

                                          Vector Space

                                          bull set of ldquovectorsrdquo

                                          bull and ldquoscalarsrdquo (coefficients)

                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                          ( in space)

                                          eg

                                          ldquo dim Euclidrsquon spacerdquo

                                          xa

                                          i

                                          ii xa

                                          d

                                          d

                                          d xx

                                          x

                                          x

                                          x 1

                                          1

                                          d

                                          Review of Linear Algebra (Cont)

                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                          Review of Linear Algebra (Cont)

                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                          Review of Linear Algebra (Cont)

                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                          Note Planes not Through the Origin

                                          are not Subspaces

                                          (Do not Contain )00 x

                                          Review of Linear Algebra (Cont)

                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                          (all Linear Combos of them =

                                          = Containing Hyperplane

                                          through Origin)

                                          Review of Linear Algebra (Cont)

                                          Basis of Subspace Set of Vectors that

                                          bull Span ie Everything is a Lin Com of them

                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                          Review of Linear Algebra (Cont)

                                          Basis of Subspace Set of Vectors that

                                          bull Span ie Everything is a Lin Com of them

                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                          bull eg ldquoUnit Vector Basisrdquo

                                          bull Since

                                          d

                                          1

                                          0

                                          0

                                          0

                                          1

                                          0

                                          0

                                          0

                                          1

                                          1

                                          0

                                          0

                                          0

                                          1

                                          0

                                          0

                                          0

                                          1

                                          212

                                          1

                                          d

                                          d

                                          xxx

                                          x

                                          x

                                          x

                                          Review of Linear Algebra (Cont)

                                          Basis Matrix of subspace of

                                          Given a basis

                                          create matrix of columns

                                          dnvv 1

                                          nddnd

                                          n

                                          n

                                          vv

                                          vv

                                          vvB

                                          1

                                          111

                                          1

                                          Review of Linear Algebra (Cont)

                                          Then linear combo is a matrix multiplicatrsquon

                                          where

                                          n

                                          iii aBva

                                          1

                                          na

                                          a

                                          a 1

                                          Review of Linear Algebra (Cont)

                                          Then linear combo is a matrix multiplicatrsquon

                                          where

                                          Note Right Multiplication Gives

                                          Linear Combination of Column Vectors

                                          n

                                          iii aBva

                                          1

                                          na

                                          a

                                          a 1

                                          Review of Linear Algebra (Cont)

                                          Then linear combo is a matrix multiplicatrsquon

                                          where

                                          Check sizes

                                          n

                                          iii aBva

                                          1

                                          na

                                          a

                                          a 1

                                          )1()(1 nndd

                                          Review of Linear Algebra (Cont)

                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                          For matrices

                                          Define the Matrix Product

                                          mkk

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          nmm

                                          n

                                          bb

                                          bb

                                          B

                                          1

                                          111

                                          m

                                          iniik

                                          m

                                          iiik

                                          m

                                          inii

                                          m

                                          iii

                                          baba

                                          baba

                                          AB

                                          1

                                          11

                                          11

                                          111

                                          Review of Linear Algebra (Cont)

                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                          For matrices

                                          Define the Matrix Product

                                          (Inner Products of Rows With Columns )

                                          mkk

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          nmm

                                          n

                                          bb

                                          bb

                                          B

                                          1

                                          111

                                          m

                                          iniik

                                          m

                                          iiik

                                          m

                                          inii

                                          m

                                          iii

                                          baba

                                          baba

                                          AB

                                          1

                                          11

                                          11

                                          111

                                          A B

                                          Review of Linear Algebra (Cont)

                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                          For matrices

                                          Define the Matrix Product

                                          (Inner Products of Rows With Columns )

                                          (Composition of Linear Transformations)

                                          mkk

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          nmm

                                          n

                                          bb

                                          bb

                                          B

                                          1

                                          111

                                          m

                                          iniik

                                          m

                                          iiik

                                          m

                                          inii

                                          m

                                          iii

                                          baba

                                          baba

                                          AB

                                          1

                                          11

                                          11

                                          111

                                          A B

                                          Review of Linear Algebra (Cont)

                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                          For matrices

                                          Define the Matrix Product

                                          (Inner Products of Rows With Columns )

                                          (Composition of Linear Transformations)

                                          Often Useful to Check Sizes

                                          mkk

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          nmm

                                          n

                                          bb

                                          bb

                                          B

                                          1

                                          111

                                          m

                                          iniik

                                          m

                                          iiik

                                          m

                                          inii

                                          m

                                          iii

                                          baba

                                          baba

                                          AB

                                          1

                                          11

                                          11

                                          111

                                          nmmknk

                                          A B

                                          Review of Linear Algebra (Cont)

                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                          For matrices

                                          Define the Matrix Product

                                          (Inner Products of Rows With Columns )

                                          (Composition of Linear Transformations)

                                          Often Useful to Check Sizes

                                          mkk

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          nmm

                                          n

                                          bb

                                          bb

                                          B

                                          1

                                          111

                                          m

                                          iniik

                                          m

                                          iiik

                                          m

                                          inii

                                          m

                                          iii

                                          baba

                                          baba

                                          AB

                                          1

                                          11

                                          11

                                          111

                                          nmmknk

                                          A B

                                          Review of Linear Algebra (Cont)

                                          Matrix Trace

                                          bull For a Square Matrix

                                          bull Define

                                          mmm

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          m

                                          iiiaAtr

                                          1)(

                                          Review of Linear Algebra (Cont)

                                          Matrix Trace

                                          bull For a Square Matrix

                                          bull Define

                                          bull Trace Commutes with Matrix Multiplication

                                          mmm

                                          m

                                          aa

                                          aa

                                          A

                                          1

                                          111

                                          m

                                          iiiaAtr

                                          1)(

                                          BAtrABtr

                                          Review of Linear Algebra (Cont)

                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                          bull Number of Elements in a Basis (Unique)

                                          Review of Linear Algebra (Cont)

                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                          bull Number of Elements in a Basis (Unique)

                                          bull (Use Basis Above)

                                          bull eg dim of a line is 1

                                          bull eg dim of a plane is 2

                                          dd dim

                                          Review of Linear Algebra (Cont)

                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                          bull Number of Elements in a Basis (Unique)

                                          bull (Use Basis Above)

                                          bull eg dim of a line is 1

                                          bull eg dim of a plane is 2

                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                          (in Statistical Uses eg ANOVA)

                                          dd dim

                                          Review of Linear Algebra (Cont)

                                          Norm of a Vector

                                          bull in d 21

                                          21

                                          1

                                          2 xxxx td

                                          jj

                                          Review of Linear Algebra (Cont)

                                          Norm of a Vector

                                          bull in

                                          bull Idea length of the vector

                                          d 2121

                                          1

                                          2 xxxx td

                                          jj

                                          Review of Linear Algebra (Cont)

                                          Norm of a Vector

                                          bull in

                                          bull Idea length of the vector

                                          bull Note strange properties for high

                                          eg ldquolength of diagonal of unit cuberdquo =

                                          d 2121

                                          1

                                          2 xxxx td

                                          jj

                                          d

                                          d

                                          Review of Linear Algebra (Cont)

                                          Norm of a Vector (cont)

                                          bull Length Normalized Vector

                                          (has Length 1 thus on Surf of Unit Sphere

                                          amp is a Direction Vector)

                                          x

                                          x

                                          Review of Linear Algebra (Cont)

                                          Norm of a Vector (cont)

                                          bull Length Normalized Vector

                                          (has Length 1 thus on Surf of Unit Sphere

                                          amp is a Direction Vector)

                                          bull Define Distance as

                                          x

                                          x

                                          yxyxyxyxd t

                                          Review of Linear Algebra (Cont)

                                          Inner (Dot Scalar) Product

                                          bull for Vectors and

                                          yxyxyx td

                                          jjj

                                          1

                                          x y

                                          Review of Linear Algebra (Cont)

                                          Inner (Dot Scalar) Product

                                          bull for Vectors and

                                          bull Related to Norm via

                                          yxyxyx td

                                          jjj

                                          1

                                          x y

                                          21 xxx

                                          Review of Linear Algebra (Cont)

                                          Inner (Dot Scalar) Product (cont)

                                          bull measures ldquoangle between and rdquo as

                                          yyxx

                                          yx

                                          yx

                                          yxyxangle

                                          tt

                                          t

                                          11 cos

                                          cos

                                          x y

                                          Review of Linear Algebra (Cont)

                                          Inner (Dot Scalar) Product (cont)

                                          bull measures ldquoangle between and rdquo as

                                          bull key to Orthogonality ie Perpendiculrsquoty

                                          if and only if

                                          yyxx

                                          yx

                                          yx

                                          yxyxangle

                                          tt

                                          t

                                          11 cos

                                          cos

                                          x y

                                          yx 0 yx

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis

                                          bull All Orthogonal to each other

                                          ie for

                                          bull All have Length 1

                                          ie for

                                          nvv 1

                                          1 ii vv

                                          0 ii vv ii

                                          ni 1

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis (cont)

                                          bull Spectral Representation

                                          where

                                          nvv 1

                                          n

                                          iii vax

                                          1

                                          ii vxa

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis (cont)

                                          bull Spectral Representation

                                          where

                                          (Coefficient is Inner Product Cool Notation)

                                          nvv 1

                                          n

                                          iii vax

                                          1

                                          ii vxa

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis (cont)

                                          bull Spectral Representation

                                          where

                                          Check

                                          nvv 1

                                          n

                                          iii vax

                                          1

                                          ii vxa

                                          iii

                                          n

                                          iii

                                          n

                                          iiii avvavvavx

                                          1

                                          1

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis (cont)

                                          bull Spectral Representation

                                          where

                                          Check

                                          bull Matrix Notation where ie

                                          For the Basis Matrix

                                          nvv 1

                                          n

                                          iii vax

                                          1

                                          ii vxa

                                          iii

                                          n

                                          iii

                                          n

                                          iiii avvavvavx

                                          1

                                          1

                                          aBx Bxa tt xBa t

                                          nvvB 1

                                          Review of Linear Algebra (Cont)

                                          Orthonormal Basis (cont)

                                          bull Spectral Representation

                                          where

                                          Check

                                          bull Matrix Notation where ie

                                          is called transform of

                                          (eg Fourier or Wavelet)

                                          nvv 1

                                          n

                                          iii vax

                                          1

                                          ii vxa

                                          iii

                                          n

                                          iii

                                          n

                                          iiii avvavvavx

                                          1

                                          1

                                          aBx Bxa tt xBa t

                                          xa

                                          Review of Linear Algebra (Cont)

                                          Parseval identity for

                                          in subsp genrsquod by o n basis

                                          x

                                          nvv 1

                                          2

                                          1

                                          22

                                          1

                                          2 aavxx

                                          n

                                          ii

                                          n

                                          ii

                                          Review of Linear Algebra (Cont)

                                          Parseval identity for

                                          in subsp genrsquod by o n basis

                                          bull Pythagorean theorem

                                          bull ldquoDecomposition of Energyrdquo

                                          bull ANOVA - sums of squares

                                          x

                                          nvv 1

                                          2

                                          1

                                          22

                                          1

                                          2 aavxx

                                          n

                                          ii

                                          n

                                          ii

                                          Review of Linear Algebra (Cont)

                                          Parseval identity for

                                          in subsp genrsquod by o n basis

                                          bull Pythagorean theorem

                                          bull ldquoDecomposition of Energyrdquo

                                          bull ANOVA - sums of squares

                                          bull Transform has same length as

                                          ie ldquorotation in rdquo

                                          x

                                          nvv 1

                                          2

                                          1

                                          22

                                          1

                                          2 aavxx

                                          n

                                          ii

                                          n

                                          ii

                                          a xd

                                          Projection of a Vector onto a Subspace

                                          bull Idea Member of that is Closest to

                                          (ie ldquoBest Approxrsquonrdquo)

                                          Review of Linear Algebra (Cont)x

                                          xV

                                          V

                                          Projection of a Vector onto a Subspace

                                          bull Idea Member of that is Closest to

                                          (ie ldquoBest Approxrsquonrdquo)

                                          bull Find that Solves

                                          (ldquoLeast Squaresrdquo)

                                          Review of Linear Algebra (Cont)x

                                          xV

                                          V

                                          VxPV vxVv

                                          min

                                          Projection of a Vector onto a Subspace

                                          bull Idea Member of that is Closest to

                                          (ie ldquoBest Approxrsquonrdquo)

                                          bull Find that Solves

                                          (ldquoLeast Squaresrdquo)

                                          bull For Inner Product (Hilbert) Space

                                          Exists and is Unique

                                          Review of Linear Algebra (Cont)x

                                          xV

                                          V

                                          VxPV vxVv

                                          min

                                          xPV

                                          Projection of a Vector onto a Subspace (cont)

                                          bull General Solution in for Basis Matrix

                                          Review of Linear Algebra (Cont)

                                          d VB

                                          xBBBBxP tVV

                                          tVVV

                                          1

                                          Projection of a Vector onto a Subspace (cont)

                                          bull General Solution in for Basis Matrix

                                          bull So Projrsquon Operator is Matrix Multrsquon

                                          (thus projection is another linear operation)

                                          Review of Linear Algebra (Cont)

                                          d VB

                                          xBBBBxP tVV

                                          tVVV

                                          1

                                          tVV

                                          tVVV BBBBP

                                          1

                                          Projection of a Vector onto a Subspace (cont)

                                          bull General Solution in for Basis Matrix

                                          bull So Projrsquon Operator is Matrix Multrsquon

                                          (thus projection is another linear operation)

                                          (note same operation underlies least squares)

                                          Review of Linear Algebra (Cont)

                                          d VB

                                          xBBBBxP tVV

                                          tVVV

                                          1

                                          tVV

                                          tVVV BBBBP

                                          1

                                          Review of Linear Algebra (Cont)

                                          Projection using Orthonormal Basis

                                          bull Basis Matrix is Orthonormal

                                          nnVtV IBB

                                          10

                                          01

                                          1

                                          111

                                          1

                                          1

                                          nnn

                                          n

                                          ntn

                                          t

                                          vvvv

                                          vvvv

                                          vv

                                          v

                                          v

                                          nvv 1

                                          Review of Linear Algebra (Cont)

                                          Projection using Orthonormal Basis

                                          bull Basis Matrix is Orthonormal

                                          bull So =

                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                          (Recall Right Multrsquon)

                                          nnVtV IBB

                                          10

                                          01

                                          1

                                          111

                                          1

                                          1

                                          nnn

                                          n

                                          ntn

                                          t

                                          vvvv

                                          vvvv

                                          vv

                                          v

                                          v

                                          xBBxP tVVV

                                          x V

                                          nvv 1

                                          Review of Linear Algebra (Cont)

                                          Projection using Orthonormal Basis (cont)

                                          bull For Orthogonal Complement

                                          and

                                          V

                                          xPxPx VV 222xPxPx VV

                                          Review of Linear Algebra (Cont)

                                          Projection using Orthonormal Basis (cont)

                                          bull For Orthogonal Complement

                                          and

                                          bull Parseval Inequality

                                          V

                                          xPxPx VV 222xPxPx VV

                                          2

                                          1

                                          22

                                          1

                                          22 aavxxxP

                                          n

                                          ii

                                          n

                                          iiV

                                          Review of Linear Algebra (Cont)

                                          (Real) Unitary Matrices with

                                          bull Orthonormal Basis Matrix

                                          (So All of Above Applies)

                                          ddU IUU t

                                          Review of Linear Algebra (Cont)

                                          (Real) Unitary Matrices with

                                          bull Orthonormal Basis Matrix

                                          (So All of Above Applies)

                                          bull Note Transformrsquon is Distance Preserving

                                          ddU IUU t

                                          yxdyxyxyxUyUxUdn

                                          i ii 2

                                          1

                                          Review of Linear Algebra (Cont)

                                          (Real) Unitary Matrices with

                                          bull Orthonormal Basis Matrix

                                          (So All of Above Applies)

                                          bull Note Transformrsquon is Distance Preserving

                                          bull Lin Trans (Mult by ) is ~ Rotation

                                          bull But also Includes ldquoMirror Imagesrdquo

                                          ddU IUU t

                                          yxdyxyxyxUyUxUdn

                                          i ii 2

                                          1

                                          U

                                          Review of Linear Algebra (Cont)

                                          Singular Value Decomposition (SVD)

                                          For a Matrix

                                          Find

                                          ndX

                                          Review of Linear Algebra (Cont)

                                          Singular Value Decomposition (SVD)

                                          For a Matrix

                                          Find a Diagonal Matrix

                                          with Entries

                                          called Singular Values

                                          ndX

                                          ndS

                                          )min(1 ndss

                                          Review of Linear Algebra (Cont)

                                          Singular Value Decomposition (SVD)

                                          For a Matrix

                                          Find a Diagonal Matrix

                                          with Entries

                                          called Singular Values

                                          And Unitary (Rotation) Matrices

                                          (recall )

                                          ndX

                                          ndS

                                          )min(1 ndss

                                          ddU nnV

                                          IVVUU tt

                                          Review of Linear Algebra (Cont)

                                          Singular Value Decomposition (SVD)

                                          For a Matrix

                                          Find a Diagonal Matrix

                                          with Entries

                                          called Singular Values

                                          And Unitary (Rotation) Matrices

                                          (recall )

                                          So That

                                          ndX

                                          ndS

                                          )min(1 ndss

                                          ddU nnV

                                          IVVUU tt tUSVX

                                          Review of Linear Algebra (Cont)

                                          Intuition behind Singular Value Decomposition

                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                          vVSUvVSUvX tt

                                          Review of Linear Algebra (Cont)

                                          Intuition behind Singular Value Decomposition

                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                          bull First rotate

                                          X

                                          vVSUvVSUvX tt

                                          Review of Linear Algebra (Cont)

                                          Intuition behind Singular Value Decomposition

                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                          bull First rotate

                                          bull Second rescale coordinate axes (by )

                                          X

                                          vVSUvVSUvX tt

                                          is

                                          Review of Linear Algebra (Cont)

                                          Intuition behind Singular Value Decomposition

                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                          bull First rotate

                                          bull Second rescale coordinate axes (by )

                                          bull Third rotate again

                                          X

                                          vVSUvVSUvX tt

                                          is

                                          Review of Linear Algebra (Cont)

                                          Intuition behind Singular Value Decomposition

                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                          bull First rotate

                                          bull Second rescale coordinate axes (by )

                                          bull Third rotate again

                                          bull ie have diagonalized the transformation

                                          X

                                          vVSUvVSUvX tt

                                          is

                                          SVD Compact Representation

                                          Useful Labeling

                                          Singular Values in Increasing Order

                                          Review of Linear Algebra (Cont)

                                          )min(1 dnss

                                          SVD Compact Representation

                                          Useful Labeling

                                          Singular Values in Increasing Order

                                          Note singular values = 0 can be omitted

                                          (Since do ldquo0-Stretchingrdquo)

                                          Review of Linear Algebra (Cont)

                                          )min(1 dnss

                                          Review of Linear Algebra (Cont)

                                          r

                                          SVD Compact Representation

                                          Useful Labeling

                                          Singular Values in Increasing Order

                                          Note singular values = 0 can be omitted

                                          Let = of positive singular values

                                          )min(1 dnss

                                          Review of Linear Algebra (Cont)

                                          r

                                          SVD Compact Representation

                                          Useful Labeling

                                          Singular Values in Increasing Order

                                          Note singular values = 0 can be omitted

                                          Let = of positive singular values

                                          Then

                                          Where are truncations of

                                          trnrrrd VSUX

                                          VSU

                                          )min(1 dnss

                                          Review of Linear Algebra (Cont)

                                          SVD Full Representation

                                          =

                                          Graphics Display Assumes

                                          ndX ddU ndS nn

                                          tV

                                          nd

                                          Review of Linear Algebra (Cont)

                                          SVD Full Representation

                                          =

                                          Full Rank Basis Matrix

                                          ndX ddU ndS nn

                                          tV

                                          Review of Linear Algebra (Cont)

                                          SVD Full Representation

                                          =

                                          Full Rank Basis Matrix

                                          All 0s in Bottom

                                          ndX ddU ndS nn

                                          tV

                                          Review of Linear Algebra (Cont)

                                          SVD Reduced Representation

                                          =

                                          These Columns Get 0ed Out

                                          ndX ddU nnS nn

                                          tV

                                          nnd 0

                                          Review of Linear Algebra (Cont)

                                          SVD Reduced Representation

                                          =ndX ndU nnS nn

                                          tV

                                          Review of Linear Algebra (Cont)

                                          SVD Reduced Representation

                                          =

                                          Also Some of These May be 0

                                          ndX ndU nnS nn

                                          tV

                                          Review of Linear Algebra (Cont)

                                          SVD Compact Representation

                                          = ndX rdU

                                          rrS nrtV

                                          0

                                          Review of Linear Algebra (Cont)

                                          SVD Compact Representation

                                          =

                                          These Get 0ed Out

                                          ndX rdU

                                          rrS nrtV

                                          0

                                          Review of Linear Algebra (Cont)

                                          SVD Compact Representation

                                          = ndX rdU

                                          rrS nrtV

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition

                                          For a (Symmetric) Square Matrix

                                          Find

                                          ddX

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition

                                          For a (Symmetric) Square Matrix

                                          Find a Diagonal Matrix

                                          Called Eigenvalues

                                          Convenient Ordering

                                          ddX

                                          d

                                          D

                                          0

                                          01

                                          n 1

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition

                                          For a (Symmetric) Square Matrix

                                          Find a Diagonal Matrix

                                          And an Orthonormal Matrix

                                          (ie )

                                          ddX

                                          d

                                          D

                                          0

                                          01

                                          ddB

                                          ddtt IBBBB

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition

                                          For a (Symmetric) Square Matrix

                                          Find a Diagonal Matrix

                                          And an Orthonormal Matrix

                                          (ie )

                                          So that ie

                                          ddX

                                          d

                                          D

                                          0

                                          01

                                          ddB

                                          ddtt IBBBB

                                          DBBX tBDBX

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                          (looks similar)

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                          Complex (uses )

                                          VU

                                          1i

                                          Review of Linear Algebra (Cont)

                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                          VU

                                          X

                                          BVU

                                          1i

                                          • Participant Presentations
                                          • PCA to find clusters
                                          • Statistical Smoothing
                                          • Kernel Density Estimation
                                          • Scatterplot Smoothing
                                          • Statistical Smoothing (2)
                                          • SiZer Background
                                          • SiZer Background (2)
                                          • SiZer Background (3)
                                          • SiZer Background (4)
                                          • SiZer Background (5)
                                          • SiZer Overview
                                          • PCA to find clusters (2)
                                          • PCA to find clusters (3)
                                          • PCA to find clusters (4)
                                          • PCA to find clusters (5)
                                          • PCA to find clusters (6)
                                          • PCA to find clusters (7)
                                          • Recall Yeast Cell Cycle Data
                                          • Yeast Cell Cycle Data FDA View
                                          • Yeast Cell Cycle Data FDA View (2)
                                          • Yeast Cell Cycle Data FDA View (3)
                                          • Yeast Cell Cycles Freq 2 Proj
                                          • Frequency 2 Analysis
                                          • Frequency 2 Analysis (2)
                                          • Frequency 2 Analysis (3)
                                          • Yeast Cell Cycle
                                          • SiZer Study of Distrsquon of Angles
                                          • Reclassification of Major Genes
                                          • Compare to Previous Classifrsquon
                                          • New Subpopulation View
                                          • New Subpopulation View (2)
                                          • Detailed Look at PCA
                                          • PCA Rediscovery ndash Renaming
                                          • PCA Rediscovery ndash Renaming (2)
                                          • PCA Rediscovery ndash Renaming (3)
                                          • PCA Rediscovery ndash Renaming (4)
                                          • PCA Rediscovery ndash Renaming (5)
                                          • An Interesting Historical Note
                                          • An Interesting Historical Note (2)
                                          • Detailed Look at PCA (2)
                                          • Detailed Look at PCA (3)
                                          • Review of Linear Algebra
                                          • Review of Linear Algebra (2)
                                          • Review of Linear Algebra (3)
                                          • Review of Linear Algebra (Cont)
                                          • Review of Linear Algebra (Cont) (2)
                                          • Review of Linear Algebra (Cont) (3)
                                          • Review of Linear Algebra (Cont) (4)
                                          • Review of Linear Algebra (Cont) (5)
                                          • Review of Linear Algebra (Cont) (6)
                                          • Review of Linear Algebra (Cont) (7)
                                          • Review of Linear Algebra (Cont) (8)
                                          • Review of Linear Algebra (Cont) (9)
                                          • Review of Linear Algebra (Cont) (10)
                                          • Review of Linear Algebra (Cont) (11)
                                          • Review of Linear Algebra (Cont) (12)
                                          • Review of Linear Algebra (Cont) (13)
                                          • Review of Linear Algebra (Cont) (14)
                                          • Review of Linear Algebra (Cont) (15)
                                          • Review of Linear Algebra (Cont) (16)
                                          • Review of Linear Algebra (Cont) (17)
                                          • Review of Linear Algebra (Cont) (18)
                                          • Review of Linear Algebra (Cont) (19)
                                          • Review of Linear Algebra (Cont) (20)
                                          • Review of Linear Algebra (Cont) (21)
                                          • Review of Linear Algebra (Cont) (22)
                                          • Review of Linear Algebra (Cont) (23)
                                          • Review of Linear Algebra (Cont) (24)
                                          • Review of Linear Algebra (Cont) (25)
                                          • Review of Linear Algebra (Cont) (26)
                                          • Review of Linear Algebra (Cont) (27)
                                          • Review of Linear Algebra (Cont) (28)
                                          • Review of Linear Algebra (Cont) (29)
                                          • Review of Linear Algebra (Cont) (30)
                                          • Review of Linear Algebra (Cont) (31)
                                          • Review of Linear Algebra (Cont) (32)
                                          • Review of Linear Algebra (Cont) (33)
                                          • Review of Linear Algebra (Cont) (34)
                                          • Review of Linear Algebra (Cont) (35)
                                          • Review of Linear Algebra (Cont) (36)
                                          • Review of Linear Algebra (Cont) (37)
                                          • Review of Linear Algebra (Cont) (38)
                                          • Review of Linear Algebra (Cont) (39)
                                          • Review of Linear Algebra (Cont) (40)
                                          • Review of Linear Algebra (Cont) (41)
                                          • Review of Linear Algebra (Cont) (42)
                                          • Review of Linear Algebra (Cont) (43)
                                          • Review of Linear Algebra (Cont) (44)
                                          • Review of Linear Algebra (Cont) (45)
                                          • Review of Linear Algebra (Cont) (46)
                                          • Review of Linear Algebra (Cont) (47)
                                          • Review of Linear Algebra (Cont) (48)
                                          • Review of Linear Algebra (Cont) (49)
                                          • Review of Linear Algebra (Cont) (50)
                                          • Review of Linear Algebra (Cont) (51)
                                          • Review of Linear Algebra (Cont) (52)
                                          • Review of Linear Algebra (Cont) (53)
                                          • Review of Linear Algebra (Cont) (54)
                                          • Review of Linear Algebra (Cont) (55)
                                          • Review of Linear Algebra (Cont) (56)
                                          • Review of Linear Algebra (Cont) (57)
                                          • Review of Linear Algebra (Cont) (58)
                                          • Review of Linear Algebra (Cont) (59)
                                          • Review of Linear Algebra (Cont) (60)
                                          • Review of Linear Algebra (Cont) (61)
                                          • Review of Linear Algebra (Cont) (62)
                                          • Review of Linear Algebra (Cont) (63)
                                          • Review of Linear Algebra (Cont) (64)
                                          • Review of Linear Algebra (Cont) (65)
                                          • Review of Linear Algebra (Cont) (66)
                                          • Review of Linear Algebra (Cont) (67)
                                          • Review of Linear Algebra (Cont) (68)
                                          • Review of Linear Algebra (Cont) (69)
                                          • Review of Linear Algebra (Cont) (70)
                                          • Review of Linear Algebra (Cont) (71)
                                          • Review of Linear Algebra (Cont) (72)
                                          • Review of Linear Algebra (Cont) (73)
                                          • Review of Linear Algebra (Cont) (74)
                                          • Review of Linear Algebra (Cont) (75)
                                          • Review of Linear Algebra (Cont) (76)
                                          • Review of Linear Algebra (Cont) (77)
                                          • Review of Linear Algebra (Cont) (78)
                                          • Review of Linear Algebra (Cont) (79)
                                          • Review of Linear Algebra (Cont) (80)
                                          • Review of Linear Algebra (Cont) (81)

                                            Yeast Cell Cycle Data FDA Viewbull Central question which genes are

                                            ldquoperiodicrdquo over 2 cell cyclesbull Naiumlve approach Simple PCAbull No apparent (2 cycle) periodic structurebull Eigenvalues suggest large amount of

                                            ldquovariationrdquobull PCA finds ldquodirections of maximal

                                            variationrdquobull Often but not always same as

                                            ldquointeresting directionsrdquobull Here need better approach to study

                                            periodicities

                                            Yeast Cell Cycles Freq 2 Proj

                                            PCA on

                                            Freq 2

                                            Periodic

                                            Component

                                            Of Data

                                            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                            Approach from Zhao Marron amp Wells (2004)

                                            Frequency 2 Analysis

                                            Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                            cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                            classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                            notable differencesbull Now try to improve ldquophase classificationrdquo

                                            Yeast Cell CycleRevisit ldquophase classificationrdquo

                                            approachbull Use outer 200 genes

                                            (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                            (finds significant bumps etc in histogram)

                                            bull Carefully redrew boundariesbull Check by studying kde angles

                                            SiZer Study of Distrsquon of Angles

                                            Reclassification of Major Genes

                                            Compare to Previous Classifrsquon

                                            New Subpopulation View

                                            New Subpopulation View

                                            NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                            Detailed Look at PCA

                                            Now Study ldquoFolklorerdquo More Carefully

                                            bull BackGround

                                            bull History

                                            bull Underpinnings

                                            (Mathematical amp Computational)

                                            Good Overall Reference Jolliffe (2002)

                                            PCA Rediscovery ndash Renaming

                                            Statistics Principal Component Analysis (PCA)

                                            PCA Rediscovery ndash Renaming

                                            Statistics Principal Component Analysis (PCA)

                                            Social Sciences Factor Analysis (PCA is a subset)

                                            PCA Rediscovery ndash Renaming

                                            Statistics Principal Component Analysis (PCA)

                                            Social Sciences Factor Analysis (PCA is a subset)

                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                            PCA Rediscovery ndash Renaming

                                            Statistics Principal Component Analysis (PCA)

                                            Social Sciences Factor Analysis (PCA is a subset)

                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                            Applied MathematicsProper Orthogonal Decomposition (POD)

                                            PCA Rediscovery ndash Renaming

                                            Statistics Principal Component Analysis (PCA)

                                            Social Sciences Factor Analysis (PCA is a subset)

                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                            Applied MathematicsProper Orthogonal Decomposition (POD)

                                            Geo-Sciences Empirical Orthogonal Functions (EOF)

                                            An Interesting Historical Note

                                            The 1st () application of PCA to Functional

                                            Data Analysis

                                            An Interesting Historical Note

                                            The 1st () application of PCA to Functional

                                            Data Analysis

                                            Rao (1958)

                                            1st Paper with ldquoCurves as Data Objectsrdquo

                                            viewpoint

                                            Detailed Look at PCA

                                            Three Important (amp Interesting) Viewpoints

                                            1 Mathematics

                                            2 Numerics

                                            3 Statistics

                                            Goal Study Interrelationships

                                            Detailed Look at PCA

                                            Three Important (amp Interesting) Viewpoints

                                            1 Mathematics

                                            2 Numerics

                                            3 Statistics

                                            1st Review Linear Alg and Multivar Prob

                                            Review of Linear Algebra

                                            Vector Space

                                            bull set of ldquovectorsrdquo

                                            bull and ldquoscalarsrdquo (coefficients)

                                            xa

                                            Review of Linear Algebra

                                            Vector Space

                                            bull set of ldquovectorsrdquo

                                            bull and ldquoscalarsrdquo (coefficients)

                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                            ( in space)

                                            xa

                                            i

                                            ii xa

                                            Review of Linear Algebra

                                            Vector Space

                                            bull set of ldquovectorsrdquo

                                            bull and ldquoscalarsrdquo (coefficients)

                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                            ( in space)

                                            eg

                                            ldquo dim Euclidrsquon spacerdquo

                                            xa

                                            i

                                            ii xa

                                            d

                                            d

                                            d xx

                                            x

                                            x

                                            x 1

                                            1

                                            d

                                            Review of Linear Algebra (Cont)

                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                            Review of Linear Algebra (Cont)

                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                            Review of Linear Algebra (Cont)

                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                            Note Planes not Through the Origin

                                            are not Subspaces

                                            (Do not Contain )00 x

                                            Review of Linear Algebra (Cont)

                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                            (all Linear Combos of them =

                                            = Containing Hyperplane

                                            through Origin)

                                            Review of Linear Algebra (Cont)

                                            Basis of Subspace Set of Vectors that

                                            bull Span ie Everything is a Lin Com of them

                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                            Review of Linear Algebra (Cont)

                                            Basis of Subspace Set of Vectors that

                                            bull Span ie Everything is a Lin Com of them

                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                            bull eg ldquoUnit Vector Basisrdquo

                                            bull Since

                                            d

                                            1

                                            0

                                            0

                                            0

                                            1

                                            0

                                            0

                                            0

                                            1

                                            1

                                            0

                                            0

                                            0

                                            1

                                            0

                                            0

                                            0

                                            1

                                            212

                                            1

                                            d

                                            d

                                            xxx

                                            x

                                            x

                                            x

                                            Review of Linear Algebra (Cont)

                                            Basis Matrix of subspace of

                                            Given a basis

                                            create matrix of columns

                                            dnvv 1

                                            nddnd

                                            n

                                            n

                                            vv

                                            vv

                                            vvB

                                            1

                                            111

                                            1

                                            Review of Linear Algebra (Cont)

                                            Then linear combo is a matrix multiplicatrsquon

                                            where

                                            n

                                            iii aBva

                                            1

                                            na

                                            a

                                            a 1

                                            Review of Linear Algebra (Cont)

                                            Then linear combo is a matrix multiplicatrsquon

                                            where

                                            Note Right Multiplication Gives

                                            Linear Combination of Column Vectors

                                            n

                                            iii aBva

                                            1

                                            na

                                            a

                                            a 1

                                            Review of Linear Algebra (Cont)

                                            Then linear combo is a matrix multiplicatrsquon

                                            where

                                            Check sizes

                                            n

                                            iii aBva

                                            1

                                            na

                                            a

                                            a 1

                                            )1()(1 nndd

                                            Review of Linear Algebra (Cont)

                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                            For matrices

                                            Define the Matrix Product

                                            mkk

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            nmm

                                            n

                                            bb

                                            bb

                                            B

                                            1

                                            111

                                            m

                                            iniik

                                            m

                                            iiik

                                            m

                                            inii

                                            m

                                            iii

                                            baba

                                            baba

                                            AB

                                            1

                                            11

                                            11

                                            111

                                            Review of Linear Algebra (Cont)

                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                            For matrices

                                            Define the Matrix Product

                                            (Inner Products of Rows With Columns )

                                            mkk

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            nmm

                                            n

                                            bb

                                            bb

                                            B

                                            1

                                            111

                                            m

                                            iniik

                                            m

                                            iiik

                                            m

                                            inii

                                            m

                                            iii

                                            baba

                                            baba

                                            AB

                                            1

                                            11

                                            11

                                            111

                                            A B

                                            Review of Linear Algebra (Cont)

                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                            For matrices

                                            Define the Matrix Product

                                            (Inner Products of Rows With Columns )

                                            (Composition of Linear Transformations)

                                            mkk

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            nmm

                                            n

                                            bb

                                            bb

                                            B

                                            1

                                            111

                                            m

                                            iniik

                                            m

                                            iiik

                                            m

                                            inii

                                            m

                                            iii

                                            baba

                                            baba

                                            AB

                                            1

                                            11

                                            11

                                            111

                                            A B

                                            Review of Linear Algebra (Cont)

                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                            For matrices

                                            Define the Matrix Product

                                            (Inner Products of Rows With Columns )

                                            (Composition of Linear Transformations)

                                            Often Useful to Check Sizes

                                            mkk

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            nmm

                                            n

                                            bb

                                            bb

                                            B

                                            1

                                            111

                                            m

                                            iniik

                                            m

                                            iiik

                                            m

                                            inii

                                            m

                                            iii

                                            baba

                                            baba

                                            AB

                                            1

                                            11

                                            11

                                            111

                                            nmmknk

                                            A B

                                            Review of Linear Algebra (Cont)

                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                            For matrices

                                            Define the Matrix Product

                                            (Inner Products of Rows With Columns )

                                            (Composition of Linear Transformations)

                                            Often Useful to Check Sizes

                                            mkk

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            nmm

                                            n

                                            bb

                                            bb

                                            B

                                            1

                                            111

                                            m

                                            iniik

                                            m

                                            iiik

                                            m

                                            inii

                                            m

                                            iii

                                            baba

                                            baba

                                            AB

                                            1

                                            11

                                            11

                                            111

                                            nmmknk

                                            A B

                                            Review of Linear Algebra (Cont)

                                            Matrix Trace

                                            bull For a Square Matrix

                                            bull Define

                                            mmm

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            m

                                            iiiaAtr

                                            1)(

                                            Review of Linear Algebra (Cont)

                                            Matrix Trace

                                            bull For a Square Matrix

                                            bull Define

                                            bull Trace Commutes with Matrix Multiplication

                                            mmm

                                            m

                                            aa

                                            aa

                                            A

                                            1

                                            111

                                            m

                                            iiiaAtr

                                            1)(

                                            BAtrABtr

                                            Review of Linear Algebra (Cont)

                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                            bull Number of Elements in a Basis (Unique)

                                            Review of Linear Algebra (Cont)

                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                            bull Number of Elements in a Basis (Unique)

                                            bull (Use Basis Above)

                                            bull eg dim of a line is 1

                                            bull eg dim of a plane is 2

                                            dd dim

                                            Review of Linear Algebra (Cont)

                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                            bull Number of Elements in a Basis (Unique)

                                            bull (Use Basis Above)

                                            bull eg dim of a line is 1

                                            bull eg dim of a plane is 2

                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                            (in Statistical Uses eg ANOVA)

                                            dd dim

                                            Review of Linear Algebra (Cont)

                                            Norm of a Vector

                                            bull in d 21

                                            21

                                            1

                                            2 xxxx td

                                            jj

                                            Review of Linear Algebra (Cont)

                                            Norm of a Vector

                                            bull in

                                            bull Idea length of the vector

                                            d 2121

                                            1

                                            2 xxxx td

                                            jj

                                            Review of Linear Algebra (Cont)

                                            Norm of a Vector

                                            bull in

                                            bull Idea length of the vector

                                            bull Note strange properties for high

                                            eg ldquolength of diagonal of unit cuberdquo =

                                            d 2121

                                            1

                                            2 xxxx td

                                            jj

                                            d

                                            d

                                            Review of Linear Algebra (Cont)

                                            Norm of a Vector (cont)

                                            bull Length Normalized Vector

                                            (has Length 1 thus on Surf of Unit Sphere

                                            amp is a Direction Vector)

                                            x

                                            x

                                            Review of Linear Algebra (Cont)

                                            Norm of a Vector (cont)

                                            bull Length Normalized Vector

                                            (has Length 1 thus on Surf of Unit Sphere

                                            amp is a Direction Vector)

                                            bull Define Distance as

                                            x

                                            x

                                            yxyxyxyxd t

                                            Review of Linear Algebra (Cont)

                                            Inner (Dot Scalar) Product

                                            bull for Vectors and

                                            yxyxyx td

                                            jjj

                                            1

                                            x y

                                            Review of Linear Algebra (Cont)

                                            Inner (Dot Scalar) Product

                                            bull for Vectors and

                                            bull Related to Norm via

                                            yxyxyx td

                                            jjj

                                            1

                                            x y

                                            21 xxx

                                            Review of Linear Algebra (Cont)

                                            Inner (Dot Scalar) Product (cont)

                                            bull measures ldquoangle between and rdquo as

                                            yyxx

                                            yx

                                            yx

                                            yxyxangle

                                            tt

                                            t

                                            11 cos

                                            cos

                                            x y

                                            Review of Linear Algebra (Cont)

                                            Inner (Dot Scalar) Product (cont)

                                            bull measures ldquoangle between and rdquo as

                                            bull key to Orthogonality ie Perpendiculrsquoty

                                            if and only if

                                            yyxx

                                            yx

                                            yx

                                            yxyxangle

                                            tt

                                            t

                                            11 cos

                                            cos

                                            x y

                                            yx 0 yx

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis

                                            bull All Orthogonal to each other

                                            ie for

                                            bull All have Length 1

                                            ie for

                                            nvv 1

                                            1 ii vv

                                            0 ii vv ii

                                            ni 1

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis (cont)

                                            bull Spectral Representation

                                            where

                                            nvv 1

                                            n

                                            iii vax

                                            1

                                            ii vxa

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis (cont)

                                            bull Spectral Representation

                                            where

                                            (Coefficient is Inner Product Cool Notation)

                                            nvv 1

                                            n

                                            iii vax

                                            1

                                            ii vxa

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis (cont)

                                            bull Spectral Representation

                                            where

                                            Check

                                            nvv 1

                                            n

                                            iii vax

                                            1

                                            ii vxa

                                            iii

                                            n

                                            iii

                                            n

                                            iiii avvavvavx

                                            1

                                            1

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis (cont)

                                            bull Spectral Representation

                                            where

                                            Check

                                            bull Matrix Notation where ie

                                            For the Basis Matrix

                                            nvv 1

                                            n

                                            iii vax

                                            1

                                            ii vxa

                                            iii

                                            n

                                            iii

                                            n

                                            iiii avvavvavx

                                            1

                                            1

                                            aBx Bxa tt xBa t

                                            nvvB 1

                                            Review of Linear Algebra (Cont)

                                            Orthonormal Basis (cont)

                                            bull Spectral Representation

                                            where

                                            Check

                                            bull Matrix Notation where ie

                                            is called transform of

                                            (eg Fourier or Wavelet)

                                            nvv 1

                                            n

                                            iii vax

                                            1

                                            ii vxa

                                            iii

                                            n

                                            iii

                                            n

                                            iiii avvavvavx

                                            1

                                            1

                                            aBx Bxa tt xBa t

                                            xa

                                            Review of Linear Algebra (Cont)

                                            Parseval identity for

                                            in subsp genrsquod by o n basis

                                            x

                                            nvv 1

                                            2

                                            1

                                            22

                                            1

                                            2 aavxx

                                            n

                                            ii

                                            n

                                            ii

                                            Review of Linear Algebra (Cont)

                                            Parseval identity for

                                            in subsp genrsquod by o n basis

                                            bull Pythagorean theorem

                                            bull ldquoDecomposition of Energyrdquo

                                            bull ANOVA - sums of squares

                                            x

                                            nvv 1

                                            2

                                            1

                                            22

                                            1

                                            2 aavxx

                                            n

                                            ii

                                            n

                                            ii

                                            Review of Linear Algebra (Cont)

                                            Parseval identity for

                                            in subsp genrsquod by o n basis

                                            bull Pythagorean theorem

                                            bull ldquoDecomposition of Energyrdquo

                                            bull ANOVA - sums of squares

                                            bull Transform has same length as

                                            ie ldquorotation in rdquo

                                            x

                                            nvv 1

                                            2

                                            1

                                            22

                                            1

                                            2 aavxx

                                            n

                                            ii

                                            n

                                            ii

                                            a xd

                                            Projection of a Vector onto a Subspace

                                            bull Idea Member of that is Closest to

                                            (ie ldquoBest Approxrsquonrdquo)

                                            Review of Linear Algebra (Cont)x

                                            xV

                                            V

                                            Projection of a Vector onto a Subspace

                                            bull Idea Member of that is Closest to

                                            (ie ldquoBest Approxrsquonrdquo)

                                            bull Find that Solves

                                            (ldquoLeast Squaresrdquo)

                                            Review of Linear Algebra (Cont)x

                                            xV

                                            V

                                            VxPV vxVv

                                            min

                                            Projection of a Vector onto a Subspace

                                            bull Idea Member of that is Closest to

                                            (ie ldquoBest Approxrsquonrdquo)

                                            bull Find that Solves

                                            (ldquoLeast Squaresrdquo)

                                            bull For Inner Product (Hilbert) Space

                                            Exists and is Unique

                                            Review of Linear Algebra (Cont)x

                                            xV

                                            V

                                            VxPV vxVv

                                            min

                                            xPV

                                            Projection of a Vector onto a Subspace (cont)

                                            bull General Solution in for Basis Matrix

                                            Review of Linear Algebra (Cont)

                                            d VB

                                            xBBBBxP tVV

                                            tVVV

                                            1

                                            Projection of a Vector onto a Subspace (cont)

                                            bull General Solution in for Basis Matrix

                                            bull So Projrsquon Operator is Matrix Multrsquon

                                            (thus projection is another linear operation)

                                            Review of Linear Algebra (Cont)

                                            d VB

                                            xBBBBxP tVV

                                            tVVV

                                            1

                                            tVV

                                            tVVV BBBBP

                                            1

                                            Projection of a Vector onto a Subspace (cont)

                                            bull General Solution in for Basis Matrix

                                            bull So Projrsquon Operator is Matrix Multrsquon

                                            (thus projection is another linear operation)

                                            (note same operation underlies least squares)

                                            Review of Linear Algebra (Cont)

                                            d VB

                                            xBBBBxP tVV

                                            tVVV

                                            1

                                            tVV

                                            tVVV BBBBP

                                            1

                                            Review of Linear Algebra (Cont)

                                            Projection using Orthonormal Basis

                                            bull Basis Matrix is Orthonormal

                                            nnVtV IBB

                                            10

                                            01

                                            1

                                            111

                                            1

                                            1

                                            nnn

                                            n

                                            ntn

                                            t

                                            vvvv

                                            vvvv

                                            vv

                                            v

                                            v

                                            nvv 1

                                            Review of Linear Algebra (Cont)

                                            Projection using Orthonormal Basis

                                            bull Basis Matrix is Orthonormal

                                            bull So =

                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                            (Recall Right Multrsquon)

                                            nnVtV IBB

                                            10

                                            01

                                            1

                                            111

                                            1

                                            1

                                            nnn

                                            n

                                            ntn

                                            t

                                            vvvv

                                            vvvv

                                            vv

                                            v

                                            v

                                            xBBxP tVVV

                                            x V

                                            nvv 1

                                            Review of Linear Algebra (Cont)

                                            Projection using Orthonormal Basis (cont)

                                            bull For Orthogonal Complement

                                            and

                                            V

                                            xPxPx VV 222xPxPx VV

                                            Review of Linear Algebra (Cont)

                                            Projection using Orthonormal Basis (cont)

                                            bull For Orthogonal Complement

                                            and

                                            bull Parseval Inequality

                                            V

                                            xPxPx VV 222xPxPx VV

                                            2

                                            1

                                            22

                                            1

                                            22 aavxxxP

                                            n

                                            ii

                                            n

                                            iiV

                                            Review of Linear Algebra (Cont)

                                            (Real) Unitary Matrices with

                                            bull Orthonormal Basis Matrix

                                            (So All of Above Applies)

                                            ddU IUU t

                                            Review of Linear Algebra (Cont)

                                            (Real) Unitary Matrices with

                                            bull Orthonormal Basis Matrix

                                            (So All of Above Applies)

                                            bull Note Transformrsquon is Distance Preserving

                                            ddU IUU t

                                            yxdyxyxyxUyUxUdn

                                            i ii 2

                                            1

                                            Review of Linear Algebra (Cont)

                                            (Real) Unitary Matrices with

                                            bull Orthonormal Basis Matrix

                                            (So All of Above Applies)

                                            bull Note Transformrsquon is Distance Preserving

                                            bull Lin Trans (Mult by ) is ~ Rotation

                                            bull But also Includes ldquoMirror Imagesrdquo

                                            ddU IUU t

                                            yxdyxyxyxUyUxUdn

                                            i ii 2

                                            1

                                            U

                                            Review of Linear Algebra (Cont)

                                            Singular Value Decomposition (SVD)

                                            For a Matrix

                                            Find

                                            ndX

                                            Review of Linear Algebra (Cont)

                                            Singular Value Decomposition (SVD)

                                            For a Matrix

                                            Find a Diagonal Matrix

                                            with Entries

                                            called Singular Values

                                            ndX

                                            ndS

                                            )min(1 ndss

                                            Review of Linear Algebra (Cont)

                                            Singular Value Decomposition (SVD)

                                            For a Matrix

                                            Find a Diagonal Matrix

                                            with Entries

                                            called Singular Values

                                            And Unitary (Rotation) Matrices

                                            (recall )

                                            ndX

                                            ndS

                                            )min(1 ndss

                                            ddU nnV

                                            IVVUU tt

                                            Review of Linear Algebra (Cont)

                                            Singular Value Decomposition (SVD)

                                            For a Matrix

                                            Find a Diagonal Matrix

                                            with Entries

                                            called Singular Values

                                            And Unitary (Rotation) Matrices

                                            (recall )

                                            So That

                                            ndX

                                            ndS

                                            )min(1 ndss

                                            ddU nnV

                                            IVVUU tt tUSVX

                                            Review of Linear Algebra (Cont)

                                            Intuition behind Singular Value Decomposition

                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                            vVSUvVSUvX tt

                                            Review of Linear Algebra (Cont)

                                            Intuition behind Singular Value Decomposition

                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                            bull First rotate

                                            X

                                            vVSUvVSUvX tt

                                            Review of Linear Algebra (Cont)

                                            Intuition behind Singular Value Decomposition

                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                            bull First rotate

                                            bull Second rescale coordinate axes (by )

                                            X

                                            vVSUvVSUvX tt

                                            is

                                            Review of Linear Algebra (Cont)

                                            Intuition behind Singular Value Decomposition

                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                            bull First rotate

                                            bull Second rescale coordinate axes (by )

                                            bull Third rotate again

                                            X

                                            vVSUvVSUvX tt

                                            is

                                            Review of Linear Algebra (Cont)

                                            Intuition behind Singular Value Decomposition

                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                            bull First rotate

                                            bull Second rescale coordinate axes (by )

                                            bull Third rotate again

                                            bull ie have diagonalized the transformation

                                            X

                                            vVSUvVSUvX tt

                                            is

                                            SVD Compact Representation

                                            Useful Labeling

                                            Singular Values in Increasing Order

                                            Review of Linear Algebra (Cont)

                                            )min(1 dnss

                                            SVD Compact Representation

                                            Useful Labeling

                                            Singular Values in Increasing Order

                                            Note singular values = 0 can be omitted

                                            (Since do ldquo0-Stretchingrdquo)

                                            Review of Linear Algebra (Cont)

                                            )min(1 dnss

                                            Review of Linear Algebra (Cont)

                                            r

                                            SVD Compact Representation

                                            Useful Labeling

                                            Singular Values in Increasing Order

                                            Note singular values = 0 can be omitted

                                            Let = of positive singular values

                                            )min(1 dnss

                                            Review of Linear Algebra (Cont)

                                            r

                                            SVD Compact Representation

                                            Useful Labeling

                                            Singular Values in Increasing Order

                                            Note singular values = 0 can be omitted

                                            Let = of positive singular values

                                            Then

                                            Where are truncations of

                                            trnrrrd VSUX

                                            VSU

                                            )min(1 dnss

                                            Review of Linear Algebra (Cont)

                                            SVD Full Representation

                                            =

                                            Graphics Display Assumes

                                            ndX ddU ndS nn

                                            tV

                                            nd

                                            Review of Linear Algebra (Cont)

                                            SVD Full Representation

                                            =

                                            Full Rank Basis Matrix

                                            ndX ddU ndS nn

                                            tV

                                            Review of Linear Algebra (Cont)

                                            SVD Full Representation

                                            =

                                            Full Rank Basis Matrix

                                            All 0s in Bottom

                                            ndX ddU ndS nn

                                            tV

                                            Review of Linear Algebra (Cont)

                                            SVD Reduced Representation

                                            =

                                            These Columns Get 0ed Out

                                            ndX ddU nnS nn

                                            tV

                                            nnd 0

                                            Review of Linear Algebra (Cont)

                                            SVD Reduced Representation

                                            =ndX ndU nnS nn

                                            tV

                                            Review of Linear Algebra (Cont)

                                            SVD Reduced Representation

                                            =

                                            Also Some of These May be 0

                                            ndX ndU nnS nn

                                            tV

                                            Review of Linear Algebra (Cont)

                                            SVD Compact Representation

                                            = ndX rdU

                                            rrS nrtV

                                            0

                                            Review of Linear Algebra (Cont)

                                            SVD Compact Representation

                                            =

                                            These Get 0ed Out

                                            ndX rdU

                                            rrS nrtV

                                            0

                                            Review of Linear Algebra (Cont)

                                            SVD Compact Representation

                                            = ndX rdU

                                            rrS nrtV

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition

                                            For a (Symmetric) Square Matrix

                                            Find

                                            ddX

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition

                                            For a (Symmetric) Square Matrix

                                            Find a Diagonal Matrix

                                            Called Eigenvalues

                                            Convenient Ordering

                                            ddX

                                            d

                                            D

                                            0

                                            01

                                            n 1

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition

                                            For a (Symmetric) Square Matrix

                                            Find a Diagonal Matrix

                                            And an Orthonormal Matrix

                                            (ie )

                                            ddX

                                            d

                                            D

                                            0

                                            01

                                            ddB

                                            ddtt IBBBB

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition

                                            For a (Symmetric) Square Matrix

                                            Find a Diagonal Matrix

                                            And an Orthonormal Matrix

                                            (ie )

                                            So that ie

                                            ddX

                                            d

                                            D

                                            0

                                            01

                                            ddB

                                            ddtt IBBBB

                                            DBBX tBDBX

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                            (looks similar)

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                            Complex (uses )

                                            VU

                                            1i

                                            Review of Linear Algebra (Cont)

                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                            VU

                                            X

                                            BVU

                                            1i

                                            • Participant Presentations
                                            • PCA to find clusters
                                            • Statistical Smoothing
                                            • Kernel Density Estimation
                                            • Scatterplot Smoothing
                                            • Statistical Smoothing (2)
                                            • SiZer Background
                                            • SiZer Background (2)
                                            • SiZer Background (3)
                                            • SiZer Background (4)
                                            • SiZer Background (5)
                                            • SiZer Overview
                                            • PCA to find clusters (2)
                                            • PCA to find clusters (3)
                                            • PCA to find clusters (4)
                                            • PCA to find clusters (5)
                                            • PCA to find clusters (6)
                                            • PCA to find clusters (7)
                                            • Recall Yeast Cell Cycle Data
                                            • Yeast Cell Cycle Data FDA View
                                            • Yeast Cell Cycle Data FDA View (2)
                                            • Yeast Cell Cycle Data FDA View (3)
                                            • Yeast Cell Cycles Freq 2 Proj
                                            • Frequency 2 Analysis
                                            • Frequency 2 Analysis (2)
                                            • Frequency 2 Analysis (3)
                                            • Yeast Cell Cycle
                                            • SiZer Study of Distrsquon of Angles
                                            • Reclassification of Major Genes
                                            • Compare to Previous Classifrsquon
                                            • New Subpopulation View
                                            • New Subpopulation View (2)
                                            • Detailed Look at PCA
                                            • PCA Rediscovery ndash Renaming
                                            • PCA Rediscovery ndash Renaming (2)
                                            • PCA Rediscovery ndash Renaming (3)
                                            • PCA Rediscovery ndash Renaming (4)
                                            • PCA Rediscovery ndash Renaming (5)
                                            • An Interesting Historical Note
                                            • An Interesting Historical Note (2)
                                            • Detailed Look at PCA (2)
                                            • Detailed Look at PCA (3)
                                            • Review of Linear Algebra
                                            • Review of Linear Algebra (2)
                                            • Review of Linear Algebra (3)
                                            • Review of Linear Algebra (Cont)
                                            • Review of Linear Algebra (Cont) (2)
                                            • Review of Linear Algebra (Cont) (3)
                                            • Review of Linear Algebra (Cont) (4)
                                            • Review of Linear Algebra (Cont) (5)
                                            • Review of Linear Algebra (Cont) (6)
                                            • Review of Linear Algebra (Cont) (7)
                                            • Review of Linear Algebra (Cont) (8)
                                            • Review of Linear Algebra (Cont) (9)
                                            • Review of Linear Algebra (Cont) (10)
                                            • Review of Linear Algebra (Cont) (11)
                                            • Review of Linear Algebra (Cont) (12)
                                            • Review of Linear Algebra (Cont) (13)
                                            • Review of Linear Algebra (Cont) (14)
                                            • Review of Linear Algebra (Cont) (15)
                                            • Review of Linear Algebra (Cont) (16)
                                            • Review of Linear Algebra (Cont) (17)
                                            • Review of Linear Algebra (Cont) (18)
                                            • Review of Linear Algebra (Cont) (19)
                                            • Review of Linear Algebra (Cont) (20)
                                            • Review of Linear Algebra (Cont) (21)
                                            • Review of Linear Algebra (Cont) (22)
                                            • Review of Linear Algebra (Cont) (23)
                                            • Review of Linear Algebra (Cont) (24)
                                            • Review of Linear Algebra (Cont) (25)
                                            • Review of Linear Algebra (Cont) (26)
                                            • Review of Linear Algebra (Cont) (27)
                                            • Review of Linear Algebra (Cont) (28)
                                            • Review of Linear Algebra (Cont) (29)
                                            • Review of Linear Algebra (Cont) (30)
                                            • Review of Linear Algebra (Cont) (31)
                                            • Review of Linear Algebra (Cont) (32)
                                            • Review of Linear Algebra (Cont) (33)
                                            • Review of Linear Algebra (Cont) (34)
                                            • Review of Linear Algebra (Cont) (35)
                                            • Review of Linear Algebra (Cont) (36)
                                            • Review of Linear Algebra (Cont) (37)
                                            • Review of Linear Algebra (Cont) (38)
                                            • Review of Linear Algebra (Cont) (39)
                                            • Review of Linear Algebra (Cont) (40)
                                            • Review of Linear Algebra (Cont) (41)
                                            • Review of Linear Algebra (Cont) (42)
                                            • Review of Linear Algebra (Cont) (43)
                                            • Review of Linear Algebra (Cont) (44)
                                            • Review of Linear Algebra (Cont) (45)
                                            • Review of Linear Algebra (Cont) (46)
                                            • Review of Linear Algebra (Cont) (47)
                                            • Review of Linear Algebra (Cont) (48)
                                            • Review of Linear Algebra (Cont) (49)
                                            • Review of Linear Algebra (Cont) (50)
                                            • Review of Linear Algebra (Cont) (51)
                                            • Review of Linear Algebra (Cont) (52)
                                            • Review of Linear Algebra (Cont) (53)
                                            • Review of Linear Algebra (Cont) (54)
                                            • Review of Linear Algebra (Cont) (55)
                                            • Review of Linear Algebra (Cont) (56)
                                            • Review of Linear Algebra (Cont) (57)
                                            • Review of Linear Algebra (Cont) (58)
                                            • Review of Linear Algebra (Cont) (59)
                                            • Review of Linear Algebra (Cont) (60)
                                            • Review of Linear Algebra (Cont) (61)
                                            • Review of Linear Algebra (Cont) (62)
                                            • Review of Linear Algebra (Cont) (63)
                                            • Review of Linear Algebra (Cont) (64)
                                            • Review of Linear Algebra (Cont) (65)
                                            • Review of Linear Algebra (Cont) (66)
                                            • Review of Linear Algebra (Cont) (67)
                                            • Review of Linear Algebra (Cont) (68)
                                            • Review of Linear Algebra (Cont) (69)
                                            • Review of Linear Algebra (Cont) (70)
                                            • Review of Linear Algebra (Cont) (71)
                                            • Review of Linear Algebra (Cont) (72)
                                            • Review of Linear Algebra (Cont) (73)
                                            • Review of Linear Algebra (Cont) (74)
                                            • Review of Linear Algebra (Cont) (75)
                                            • Review of Linear Algebra (Cont) (76)
                                            • Review of Linear Algebra (Cont) (77)
                                            • Review of Linear Algebra (Cont) (78)
                                            • Review of Linear Algebra (Cont) (79)
                                            • Review of Linear Algebra (Cont) (80)
                                            • Review of Linear Algebra (Cont) (81)

                                              Yeast Cell Cycles Freq 2 Proj

                                              PCA on

                                              Freq 2

                                              Periodic

                                              Component

                                              Of Data

                                              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                              Approach from Zhao Marron amp Wells (2004)

                                              Frequency 2 Analysis

                                              Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                              cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                              classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                              notable differencesbull Now try to improve ldquophase classificationrdquo

                                              Yeast Cell CycleRevisit ldquophase classificationrdquo

                                              approachbull Use outer 200 genes

                                              (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                              (finds significant bumps etc in histogram)

                                              bull Carefully redrew boundariesbull Check by studying kde angles

                                              SiZer Study of Distrsquon of Angles

                                              Reclassification of Major Genes

                                              Compare to Previous Classifrsquon

                                              New Subpopulation View

                                              New Subpopulation View

                                              NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                              Detailed Look at PCA

                                              Now Study ldquoFolklorerdquo More Carefully

                                              bull BackGround

                                              bull History

                                              bull Underpinnings

                                              (Mathematical amp Computational)

                                              Good Overall Reference Jolliffe (2002)

                                              PCA Rediscovery ndash Renaming

                                              Statistics Principal Component Analysis (PCA)

                                              PCA Rediscovery ndash Renaming

                                              Statistics Principal Component Analysis (PCA)

                                              Social Sciences Factor Analysis (PCA is a subset)

                                              PCA Rediscovery ndash Renaming

                                              Statistics Principal Component Analysis (PCA)

                                              Social Sciences Factor Analysis (PCA is a subset)

                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                              PCA Rediscovery ndash Renaming

                                              Statistics Principal Component Analysis (PCA)

                                              Social Sciences Factor Analysis (PCA is a subset)

                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                              Applied MathematicsProper Orthogonal Decomposition (POD)

                                              PCA Rediscovery ndash Renaming

                                              Statistics Principal Component Analysis (PCA)

                                              Social Sciences Factor Analysis (PCA is a subset)

                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                              Applied MathematicsProper Orthogonal Decomposition (POD)

                                              Geo-Sciences Empirical Orthogonal Functions (EOF)

                                              An Interesting Historical Note

                                              The 1st () application of PCA to Functional

                                              Data Analysis

                                              An Interesting Historical Note

                                              The 1st () application of PCA to Functional

                                              Data Analysis

                                              Rao (1958)

                                              1st Paper with ldquoCurves as Data Objectsrdquo

                                              viewpoint

                                              Detailed Look at PCA

                                              Three Important (amp Interesting) Viewpoints

                                              1 Mathematics

                                              2 Numerics

                                              3 Statistics

                                              Goal Study Interrelationships

                                              Detailed Look at PCA

                                              Three Important (amp Interesting) Viewpoints

                                              1 Mathematics

                                              2 Numerics

                                              3 Statistics

                                              1st Review Linear Alg and Multivar Prob

                                              Review of Linear Algebra

                                              Vector Space

                                              bull set of ldquovectorsrdquo

                                              bull and ldquoscalarsrdquo (coefficients)

                                              xa

                                              Review of Linear Algebra

                                              Vector Space

                                              bull set of ldquovectorsrdquo

                                              bull and ldquoscalarsrdquo (coefficients)

                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                              ( in space)

                                              xa

                                              i

                                              ii xa

                                              Review of Linear Algebra

                                              Vector Space

                                              bull set of ldquovectorsrdquo

                                              bull and ldquoscalarsrdquo (coefficients)

                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                              ( in space)

                                              eg

                                              ldquo dim Euclidrsquon spacerdquo

                                              xa

                                              i

                                              ii xa

                                              d

                                              d

                                              d xx

                                              x

                                              x

                                              x 1

                                              1

                                              d

                                              Review of Linear Algebra (Cont)

                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                              Review of Linear Algebra (Cont)

                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                              Review of Linear Algebra (Cont)

                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                              Note Planes not Through the Origin

                                              are not Subspaces

                                              (Do not Contain )00 x

                                              Review of Linear Algebra (Cont)

                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                              (all Linear Combos of them =

                                              = Containing Hyperplane

                                              through Origin)

                                              Review of Linear Algebra (Cont)

                                              Basis of Subspace Set of Vectors that

                                              bull Span ie Everything is a Lin Com of them

                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                              Review of Linear Algebra (Cont)

                                              Basis of Subspace Set of Vectors that

                                              bull Span ie Everything is a Lin Com of them

                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                              bull eg ldquoUnit Vector Basisrdquo

                                              bull Since

                                              d

                                              1

                                              0

                                              0

                                              0

                                              1

                                              0

                                              0

                                              0

                                              1

                                              1

                                              0

                                              0

                                              0

                                              1

                                              0

                                              0

                                              0

                                              1

                                              212

                                              1

                                              d

                                              d

                                              xxx

                                              x

                                              x

                                              x

                                              Review of Linear Algebra (Cont)

                                              Basis Matrix of subspace of

                                              Given a basis

                                              create matrix of columns

                                              dnvv 1

                                              nddnd

                                              n

                                              n

                                              vv

                                              vv

                                              vvB

                                              1

                                              111

                                              1

                                              Review of Linear Algebra (Cont)

                                              Then linear combo is a matrix multiplicatrsquon

                                              where

                                              n

                                              iii aBva

                                              1

                                              na

                                              a

                                              a 1

                                              Review of Linear Algebra (Cont)

                                              Then linear combo is a matrix multiplicatrsquon

                                              where

                                              Note Right Multiplication Gives

                                              Linear Combination of Column Vectors

                                              n

                                              iii aBva

                                              1

                                              na

                                              a

                                              a 1

                                              Review of Linear Algebra (Cont)

                                              Then linear combo is a matrix multiplicatrsquon

                                              where

                                              Check sizes

                                              n

                                              iii aBva

                                              1

                                              na

                                              a

                                              a 1

                                              )1()(1 nndd

                                              Review of Linear Algebra (Cont)

                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                              For matrices

                                              Define the Matrix Product

                                              mkk

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              nmm

                                              n

                                              bb

                                              bb

                                              B

                                              1

                                              111

                                              m

                                              iniik

                                              m

                                              iiik

                                              m

                                              inii

                                              m

                                              iii

                                              baba

                                              baba

                                              AB

                                              1

                                              11

                                              11

                                              111

                                              Review of Linear Algebra (Cont)

                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                              For matrices

                                              Define the Matrix Product

                                              (Inner Products of Rows With Columns )

                                              mkk

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              nmm

                                              n

                                              bb

                                              bb

                                              B

                                              1

                                              111

                                              m

                                              iniik

                                              m

                                              iiik

                                              m

                                              inii

                                              m

                                              iii

                                              baba

                                              baba

                                              AB

                                              1

                                              11

                                              11

                                              111

                                              A B

                                              Review of Linear Algebra (Cont)

                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                              For matrices

                                              Define the Matrix Product

                                              (Inner Products of Rows With Columns )

                                              (Composition of Linear Transformations)

                                              mkk

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              nmm

                                              n

                                              bb

                                              bb

                                              B

                                              1

                                              111

                                              m

                                              iniik

                                              m

                                              iiik

                                              m

                                              inii

                                              m

                                              iii

                                              baba

                                              baba

                                              AB

                                              1

                                              11

                                              11

                                              111

                                              A B

                                              Review of Linear Algebra (Cont)

                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                              For matrices

                                              Define the Matrix Product

                                              (Inner Products of Rows With Columns )

                                              (Composition of Linear Transformations)

                                              Often Useful to Check Sizes

                                              mkk

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              nmm

                                              n

                                              bb

                                              bb

                                              B

                                              1

                                              111

                                              m

                                              iniik

                                              m

                                              iiik

                                              m

                                              inii

                                              m

                                              iii

                                              baba

                                              baba

                                              AB

                                              1

                                              11

                                              11

                                              111

                                              nmmknk

                                              A B

                                              Review of Linear Algebra (Cont)

                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                              For matrices

                                              Define the Matrix Product

                                              (Inner Products of Rows With Columns )

                                              (Composition of Linear Transformations)

                                              Often Useful to Check Sizes

                                              mkk

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              nmm

                                              n

                                              bb

                                              bb

                                              B

                                              1

                                              111

                                              m

                                              iniik

                                              m

                                              iiik

                                              m

                                              inii

                                              m

                                              iii

                                              baba

                                              baba

                                              AB

                                              1

                                              11

                                              11

                                              111

                                              nmmknk

                                              A B

                                              Review of Linear Algebra (Cont)

                                              Matrix Trace

                                              bull For a Square Matrix

                                              bull Define

                                              mmm

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              m

                                              iiiaAtr

                                              1)(

                                              Review of Linear Algebra (Cont)

                                              Matrix Trace

                                              bull For a Square Matrix

                                              bull Define

                                              bull Trace Commutes with Matrix Multiplication

                                              mmm

                                              m

                                              aa

                                              aa

                                              A

                                              1

                                              111

                                              m

                                              iiiaAtr

                                              1)(

                                              BAtrABtr

                                              Review of Linear Algebra (Cont)

                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                              bull Number of Elements in a Basis (Unique)

                                              Review of Linear Algebra (Cont)

                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                              bull Number of Elements in a Basis (Unique)

                                              bull (Use Basis Above)

                                              bull eg dim of a line is 1

                                              bull eg dim of a plane is 2

                                              dd dim

                                              Review of Linear Algebra (Cont)

                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                              bull Number of Elements in a Basis (Unique)

                                              bull (Use Basis Above)

                                              bull eg dim of a line is 1

                                              bull eg dim of a plane is 2

                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                              (in Statistical Uses eg ANOVA)

                                              dd dim

                                              Review of Linear Algebra (Cont)

                                              Norm of a Vector

                                              bull in d 21

                                              21

                                              1

                                              2 xxxx td

                                              jj

                                              Review of Linear Algebra (Cont)

                                              Norm of a Vector

                                              bull in

                                              bull Idea length of the vector

                                              d 2121

                                              1

                                              2 xxxx td

                                              jj

                                              Review of Linear Algebra (Cont)

                                              Norm of a Vector

                                              bull in

                                              bull Idea length of the vector

                                              bull Note strange properties for high

                                              eg ldquolength of diagonal of unit cuberdquo =

                                              d 2121

                                              1

                                              2 xxxx td

                                              jj

                                              d

                                              d

                                              Review of Linear Algebra (Cont)

                                              Norm of a Vector (cont)

                                              bull Length Normalized Vector

                                              (has Length 1 thus on Surf of Unit Sphere

                                              amp is a Direction Vector)

                                              x

                                              x

                                              Review of Linear Algebra (Cont)

                                              Norm of a Vector (cont)

                                              bull Length Normalized Vector

                                              (has Length 1 thus on Surf of Unit Sphere

                                              amp is a Direction Vector)

                                              bull Define Distance as

                                              x

                                              x

                                              yxyxyxyxd t

                                              Review of Linear Algebra (Cont)

                                              Inner (Dot Scalar) Product

                                              bull for Vectors and

                                              yxyxyx td

                                              jjj

                                              1

                                              x y

                                              Review of Linear Algebra (Cont)

                                              Inner (Dot Scalar) Product

                                              bull for Vectors and

                                              bull Related to Norm via

                                              yxyxyx td

                                              jjj

                                              1

                                              x y

                                              21 xxx

                                              Review of Linear Algebra (Cont)

                                              Inner (Dot Scalar) Product (cont)

                                              bull measures ldquoangle between and rdquo as

                                              yyxx

                                              yx

                                              yx

                                              yxyxangle

                                              tt

                                              t

                                              11 cos

                                              cos

                                              x y

                                              Review of Linear Algebra (Cont)

                                              Inner (Dot Scalar) Product (cont)

                                              bull measures ldquoangle between and rdquo as

                                              bull key to Orthogonality ie Perpendiculrsquoty

                                              if and only if

                                              yyxx

                                              yx

                                              yx

                                              yxyxangle

                                              tt

                                              t

                                              11 cos

                                              cos

                                              x y

                                              yx 0 yx

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis

                                              bull All Orthogonal to each other

                                              ie for

                                              bull All have Length 1

                                              ie for

                                              nvv 1

                                              1 ii vv

                                              0 ii vv ii

                                              ni 1

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis (cont)

                                              bull Spectral Representation

                                              where

                                              nvv 1

                                              n

                                              iii vax

                                              1

                                              ii vxa

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis (cont)

                                              bull Spectral Representation

                                              where

                                              (Coefficient is Inner Product Cool Notation)

                                              nvv 1

                                              n

                                              iii vax

                                              1

                                              ii vxa

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis (cont)

                                              bull Spectral Representation

                                              where

                                              Check

                                              nvv 1

                                              n

                                              iii vax

                                              1

                                              ii vxa

                                              iii

                                              n

                                              iii

                                              n

                                              iiii avvavvavx

                                              1

                                              1

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis (cont)

                                              bull Spectral Representation

                                              where

                                              Check

                                              bull Matrix Notation where ie

                                              For the Basis Matrix

                                              nvv 1

                                              n

                                              iii vax

                                              1

                                              ii vxa

                                              iii

                                              n

                                              iii

                                              n

                                              iiii avvavvavx

                                              1

                                              1

                                              aBx Bxa tt xBa t

                                              nvvB 1

                                              Review of Linear Algebra (Cont)

                                              Orthonormal Basis (cont)

                                              bull Spectral Representation

                                              where

                                              Check

                                              bull Matrix Notation where ie

                                              is called transform of

                                              (eg Fourier or Wavelet)

                                              nvv 1

                                              n

                                              iii vax

                                              1

                                              ii vxa

                                              iii

                                              n

                                              iii

                                              n

                                              iiii avvavvavx

                                              1

                                              1

                                              aBx Bxa tt xBa t

                                              xa

                                              Review of Linear Algebra (Cont)

                                              Parseval identity for

                                              in subsp genrsquod by o n basis

                                              x

                                              nvv 1

                                              2

                                              1

                                              22

                                              1

                                              2 aavxx

                                              n

                                              ii

                                              n

                                              ii

                                              Review of Linear Algebra (Cont)

                                              Parseval identity for

                                              in subsp genrsquod by o n basis

                                              bull Pythagorean theorem

                                              bull ldquoDecomposition of Energyrdquo

                                              bull ANOVA - sums of squares

                                              x

                                              nvv 1

                                              2

                                              1

                                              22

                                              1

                                              2 aavxx

                                              n

                                              ii

                                              n

                                              ii

                                              Review of Linear Algebra (Cont)

                                              Parseval identity for

                                              in subsp genrsquod by o n basis

                                              bull Pythagorean theorem

                                              bull ldquoDecomposition of Energyrdquo

                                              bull ANOVA - sums of squares

                                              bull Transform has same length as

                                              ie ldquorotation in rdquo

                                              x

                                              nvv 1

                                              2

                                              1

                                              22

                                              1

                                              2 aavxx

                                              n

                                              ii

                                              n

                                              ii

                                              a xd

                                              Projection of a Vector onto a Subspace

                                              bull Idea Member of that is Closest to

                                              (ie ldquoBest Approxrsquonrdquo)

                                              Review of Linear Algebra (Cont)x

                                              xV

                                              V

                                              Projection of a Vector onto a Subspace

                                              bull Idea Member of that is Closest to

                                              (ie ldquoBest Approxrsquonrdquo)

                                              bull Find that Solves

                                              (ldquoLeast Squaresrdquo)

                                              Review of Linear Algebra (Cont)x

                                              xV

                                              V

                                              VxPV vxVv

                                              min

                                              Projection of a Vector onto a Subspace

                                              bull Idea Member of that is Closest to

                                              (ie ldquoBest Approxrsquonrdquo)

                                              bull Find that Solves

                                              (ldquoLeast Squaresrdquo)

                                              bull For Inner Product (Hilbert) Space

                                              Exists and is Unique

                                              Review of Linear Algebra (Cont)x

                                              xV

                                              V

                                              VxPV vxVv

                                              min

                                              xPV

                                              Projection of a Vector onto a Subspace (cont)

                                              bull General Solution in for Basis Matrix

                                              Review of Linear Algebra (Cont)

                                              d VB

                                              xBBBBxP tVV

                                              tVVV

                                              1

                                              Projection of a Vector onto a Subspace (cont)

                                              bull General Solution in for Basis Matrix

                                              bull So Projrsquon Operator is Matrix Multrsquon

                                              (thus projection is another linear operation)

                                              Review of Linear Algebra (Cont)

                                              d VB

                                              xBBBBxP tVV

                                              tVVV

                                              1

                                              tVV

                                              tVVV BBBBP

                                              1

                                              Projection of a Vector onto a Subspace (cont)

                                              bull General Solution in for Basis Matrix

                                              bull So Projrsquon Operator is Matrix Multrsquon

                                              (thus projection is another linear operation)

                                              (note same operation underlies least squares)

                                              Review of Linear Algebra (Cont)

                                              d VB

                                              xBBBBxP tVV

                                              tVVV

                                              1

                                              tVV

                                              tVVV BBBBP

                                              1

                                              Review of Linear Algebra (Cont)

                                              Projection using Orthonormal Basis

                                              bull Basis Matrix is Orthonormal

                                              nnVtV IBB

                                              10

                                              01

                                              1

                                              111

                                              1

                                              1

                                              nnn

                                              n

                                              ntn

                                              t

                                              vvvv

                                              vvvv

                                              vv

                                              v

                                              v

                                              nvv 1

                                              Review of Linear Algebra (Cont)

                                              Projection using Orthonormal Basis

                                              bull Basis Matrix is Orthonormal

                                              bull So =

                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                              (Recall Right Multrsquon)

                                              nnVtV IBB

                                              10

                                              01

                                              1

                                              111

                                              1

                                              1

                                              nnn

                                              n

                                              ntn

                                              t

                                              vvvv

                                              vvvv

                                              vv

                                              v

                                              v

                                              xBBxP tVVV

                                              x V

                                              nvv 1

                                              Review of Linear Algebra (Cont)

                                              Projection using Orthonormal Basis (cont)

                                              bull For Orthogonal Complement

                                              and

                                              V

                                              xPxPx VV 222xPxPx VV

                                              Review of Linear Algebra (Cont)

                                              Projection using Orthonormal Basis (cont)

                                              bull For Orthogonal Complement

                                              and

                                              bull Parseval Inequality

                                              V

                                              xPxPx VV 222xPxPx VV

                                              2

                                              1

                                              22

                                              1

                                              22 aavxxxP

                                              n

                                              ii

                                              n

                                              iiV

                                              Review of Linear Algebra (Cont)

                                              (Real) Unitary Matrices with

                                              bull Orthonormal Basis Matrix

                                              (So All of Above Applies)

                                              ddU IUU t

                                              Review of Linear Algebra (Cont)

                                              (Real) Unitary Matrices with

                                              bull Orthonormal Basis Matrix

                                              (So All of Above Applies)

                                              bull Note Transformrsquon is Distance Preserving

                                              ddU IUU t

                                              yxdyxyxyxUyUxUdn

                                              i ii 2

                                              1

                                              Review of Linear Algebra (Cont)

                                              (Real) Unitary Matrices with

                                              bull Orthonormal Basis Matrix

                                              (So All of Above Applies)

                                              bull Note Transformrsquon is Distance Preserving

                                              bull Lin Trans (Mult by ) is ~ Rotation

                                              bull But also Includes ldquoMirror Imagesrdquo

                                              ddU IUU t

                                              yxdyxyxyxUyUxUdn

                                              i ii 2

                                              1

                                              U

                                              Review of Linear Algebra (Cont)

                                              Singular Value Decomposition (SVD)

                                              For a Matrix

                                              Find

                                              ndX

                                              Review of Linear Algebra (Cont)

                                              Singular Value Decomposition (SVD)

                                              For a Matrix

                                              Find a Diagonal Matrix

                                              with Entries

                                              called Singular Values

                                              ndX

                                              ndS

                                              )min(1 ndss

                                              Review of Linear Algebra (Cont)

                                              Singular Value Decomposition (SVD)

                                              For a Matrix

                                              Find a Diagonal Matrix

                                              with Entries

                                              called Singular Values

                                              And Unitary (Rotation) Matrices

                                              (recall )

                                              ndX

                                              ndS

                                              )min(1 ndss

                                              ddU nnV

                                              IVVUU tt

                                              Review of Linear Algebra (Cont)

                                              Singular Value Decomposition (SVD)

                                              For a Matrix

                                              Find a Diagonal Matrix

                                              with Entries

                                              called Singular Values

                                              And Unitary (Rotation) Matrices

                                              (recall )

                                              So That

                                              ndX

                                              ndS

                                              )min(1 ndss

                                              ddU nnV

                                              IVVUU tt tUSVX

                                              Review of Linear Algebra (Cont)

                                              Intuition behind Singular Value Decomposition

                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                              vVSUvVSUvX tt

                                              Review of Linear Algebra (Cont)

                                              Intuition behind Singular Value Decomposition

                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                              bull First rotate

                                              X

                                              vVSUvVSUvX tt

                                              Review of Linear Algebra (Cont)

                                              Intuition behind Singular Value Decomposition

                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                              bull First rotate

                                              bull Second rescale coordinate axes (by )

                                              X

                                              vVSUvVSUvX tt

                                              is

                                              Review of Linear Algebra (Cont)

                                              Intuition behind Singular Value Decomposition

                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                              bull First rotate

                                              bull Second rescale coordinate axes (by )

                                              bull Third rotate again

                                              X

                                              vVSUvVSUvX tt

                                              is

                                              Review of Linear Algebra (Cont)

                                              Intuition behind Singular Value Decomposition

                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                              bull First rotate

                                              bull Second rescale coordinate axes (by )

                                              bull Third rotate again

                                              bull ie have diagonalized the transformation

                                              X

                                              vVSUvVSUvX tt

                                              is

                                              SVD Compact Representation

                                              Useful Labeling

                                              Singular Values in Increasing Order

                                              Review of Linear Algebra (Cont)

                                              )min(1 dnss

                                              SVD Compact Representation

                                              Useful Labeling

                                              Singular Values in Increasing Order

                                              Note singular values = 0 can be omitted

                                              (Since do ldquo0-Stretchingrdquo)

                                              Review of Linear Algebra (Cont)

                                              )min(1 dnss

                                              Review of Linear Algebra (Cont)

                                              r

                                              SVD Compact Representation

                                              Useful Labeling

                                              Singular Values in Increasing Order

                                              Note singular values = 0 can be omitted

                                              Let = of positive singular values

                                              )min(1 dnss

                                              Review of Linear Algebra (Cont)

                                              r

                                              SVD Compact Representation

                                              Useful Labeling

                                              Singular Values in Increasing Order

                                              Note singular values = 0 can be omitted

                                              Let = of positive singular values

                                              Then

                                              Where are truncations of

                                              trnrrrd VSUX

                                              VSU

                                              )min(1 dnss

                                              Review of Linear Algebra (Cont)

                                              SVD Full Representation

                                              =

                                              Graphics Display Assumes

                                              ndX ddU ndS nn

                                              tV

                                              nd

                                              Review of Linear Algebra (Cont)

                                              SVD Full Representation

                                              =

                                              Full Rank Basis Matrix

                                              ndX ddU ndS nn

                                              tV

                                              Review of Linear Algebra (Cont)

                                              SVD Full Representation

                                              =

                                              Full Rank Basis Matrix

                                              All 0s in Bottom

                                              ndX ddU ndS nn

                                              tV

                                              Review of Linear Algebra (Cont)

                                              SVD Reduced Representation

                                              =

                                              These Columns Get 0ed Out

                                              ndX ddU nnS nn

                                              tV

                                              nnd 0

                                              Review of Linear Algebra (Cont)

                                              SVD Reduced Representation

                                              =ndX ndU nnS nn

                                              tV

                                              Review of Linear Algebra (Cont)

                                              SVD Reduced Representation

                                              =

                                              Also Some of These May be 0

                                              ndX ndU nnS nn

                                              tV

                                              Review of Linear Algebra (Cont)

                                              SVD Compact Representation

                                              = ndX rdU

                                              rrS nrtV

                                              0

                                              Review of Linear Algebra (Cont)

                                              SVD Compact Representation

                                              =

                                              These Get 0ed Out

                                              ndX rdU

                                              rrS nrtV

                                              0

                                              Review of Linear Algebra (Cont)

                                              SVD Compact Representation

                                              = ndX rdU

                                              rrS nrtV

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition

                                              For a (Symmetric) Square Matrix

                                              Find

                                              ddX

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition

                                              For a (Symmetric) Square Matrix

                                              Find a Diagonal Matrix

                                              Called Eigenvalues

                                              Convenient Ordering

                                              ddX

                                              d

                                              D

                                              0

                                              01

                                              n 1

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition

                                              For a (Symmetric) Square Matrix

                                              Find a Diagonal Matrix

                                              And an Orthonormal Matrix

                                              (ie )

                                              ddX

                                              d

                                              D

                                              0

                                              01

                                              ddB

                                              ddtt IBBBB

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition

                                              For a (Symmetric) Square Matrix

                                              Find a Diagonal Matrix

                                              And an Orthonormal Matrix

                                              (ie )

                                              So that ie

                                              ddX

                                              d

                                              D

                                              0

                                              01

                                              ddB

                                              ddtt IBBBB

                                              DBBX tBDBX

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                              (looks similar)

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                              Complex (uses )

                                              VU

                                              1i

                                              Review of Linear Algebra (Cont)

                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                              VU

                                              X

                                              BVU

                                              1i

                                              • Participant Presentations
                                              • PCA to find clusters
                                              • Statistical Smoothing
                                              • Kernel Density Estimation
                                              • Scatterplot Smoothing
                                              • Statistical Smoothing (2)
                                              • SiZer Background
                                              • SiZer Background (2)
                                              • SiZer Background (3)
                                              • SiZer Background (4)
                                              • SiZer Background (5)
                                              • SiZer Overview
                                              • PCA to find clusters (2)
                                              • PCA to find clusters (3)
                                              • PCA to find clusters (4)
                                              • PCA to find clusters (5)
                                              • PCA to find clusters (6)
                                              • PCA to find clusters (7)
                                              • Recall Yeast Cell Cycle Data
                                              • Yeast Cell Cycle Data FDA View
                                              • Yeast Cell Cycle Data FDA View (2)
                                              • Yeast Cell Cycle Data FDA View (3)
                                              • Yeast Cell Cycles Freq 2 Proj
                                              • Frequency 2 Analysis
                                              • Frequency 2 Analysis (2)
                                              • Frequency 2 Analysis (3)
                                              • Yeast Cell Cycle
                                              • SiZer Study of Distrsquon of Angles
                                              • Reclassification of Major Genes
                                              • Compare to Previous Classifrsquon
                                              • New Subpopulation View
                                              • New Subpopulation View (2)
                                              • Detailed Look at PCA
                                              • PCA Rediscovery ndash Renaming
                                              • PCA Rediscovery ndash Renaming (2)
                                              • PCA Rediscovery ndash Renaming (3)
                                              • PCA Rediscovery ndash Renaming (4)
                                              • PCA Rediscovery ndash Renaming (5)
                                              • An Interesting Historical Note
                                              • An Interesting Historical Note (2)
                                              • Detailed Look at PCA (2)
                                              • Detailed Look at PCA (3)
                                              • Review of Linear Algebra
                                              • Review of Linear Algebra (2)
                                              • Review of Linear Algebra (3)
                                              • Review of Linear Algebra (Cont)
                                              • Review of Linear Algebra (Cont) (2)
                                              • Review of Linear Algebra (Cont) (3)
                                              • Review of Linear Algebra (Cont) (4)
                                              • Review of Linear Algebra (Cont) (5)
                                              • Review of Linear Algebra (Cont) (6)
                                              • Review of Linear Algebra (Cont) (7)
                                              • Review of Linear Algebra (Cont) (8)
                                              • Review of Linear Algebra (Cont) (9)
                                              • Review of Linear Algebra (Cont) (10)
                                              • Review of Linear Algebra (Cont) (11)
                                              • Review of Linear Algebra (Cont) (12)
                                              • Review of Linear Algebra (Cont) (13)
                                              • Review of Linear Algebra (Cont) (14)
                                              • Review of Linear Algebra (Cont) (15)
                                              • Review of Linear Algebra (Cont) (16)
                                              • Review of Linear Algebra (Cont) (17)
                                              • Review of Linear Algebra (Cont) (18)
                                              • Review of Linear Algebra (Cont) (19)
                                              • Review of Linear Algebra (Cont) (20)
                                              • Review of Linear Algebra (Cont) (21)
                                              • Review of Linear Algebra (Cont) (22)
                                              • Review of Linear Algebra (Cont) (23)
                                              • Review of Linear Algebra (Cont) (24)
                                              • Review of Linear Algebra (Cont) (25)
                                              • Review of Linear Algebra (Cont) (26)
                                              • Review of Linear Algebra (Cont) (27)
                                              • Review of Linear Algebra (Cont) (28)
                                              • Review of Linear Algebra (Cont) (29)
                                              • Review of Linear Algebra (Cont) (30)
                                              • Review of Linear Algebra (Cont) (31)
                                              • Review of Linear Algebra (Cont) (32)
                                              • Review of Linear Algebra (Cont) (33)
                                              • Review of Linear Algebra (Cont) (34)
                                              • Review of Linear Algebra (Cont) (35)
                                              • Review of Linear Algebra (Cont) (36)
                                              • Review of Linear Algebra (Cont) (37)
                                              • Review of Linear Algebra (Cont) (38)
                                              • Review of Linear Algebra (Cont) (39)
                                              • Review of Linear Algebra (Cont) (40)
                                              • Review of Linear Algebra (Cont) (41)
                                              • Review of Linear Algebra (Cont) (42)
                                              • Review of Linear Algebra (Cont) (43)
                                              • Review of Linear Algebra (Cont) (44)
                                              • Review of Linear Algebra (Cont) (45)
                                              • Review of Linear Algebra (Cont) (46)
                                              • Review of Linear Algebra (Cont) (47)
                                              • Review of Linear Algebra (Cont) (48)
                                              • Review of Linear Algebra (Cont) (49)
                                              • Review of Linear Algebra (Cont) (50)
                                              • Review of Linear Algebra (Cont) (51)
                                              • Review of Linear Algebra (Cont) (52)
                                              • Review of Linear Algebra (Cont) (53)
                                              • Review of Linear Algebra (Cont) (54)
                                              • Review of Linear Algebra (Cont) (55)
                                              • Review of Linear Algebra (Cont) (56)
                                              • Review of Linear Algebra (Cont) (57)
                                              • Review of Linear Algebra (Cont) (58)
                                              • Review of Linear Algebra (Cont) (59)
                                              • Review of Linear Algebra (Cont) (60)
                                              • Review of Linear Algebra (Cont) (61)
                                              • Review of Linear Algebra (Cont) (62)
                                              • Review of Linear Algebra (Cont) (63)
                                              • Review of Linear Algebra (Cont) (64)
                                              • Review of Linear Algebra (Cont) (65)
                                              • Review of Linear Algebra (Cont) (66)
                                              • Review of Linear Algebra (Cont) (67)
                                              • Review of Linear Algebra (Cont) (68)
                                              • Review of Linear Algebra (Cont) (69)
                                              • Review of Linear Algebra (Cont) (70)
                                              • Review of Linear Algebra (Cont) (71)
                                              • Review of Linear Algebra (Cont) (72)
                                              • Review of Linear Algebra (Cont) (73)
                                              • Review of Linear Algebra (Cont) (74)
                                              • Review of Linear Algebra (Cont) (75)
                                              • Review of Linear Algebra (Cont) (76)
                                              • Review of Linear Algebra (Cont) (77)
                                              • Review of Linear Algebra (Cont) (78)
                                              • Review of Linear Algebra (Cont) (79)
                                              • Review of Linear Algebra (Cont) (80)
                                              • Review of Linear Algebra (Cont) (81)

                                                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phase

                                                Approach from Zhao Marron amp Wells (2004)

                                                Frequency 2 Analysis

                                                Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                                cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                                classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                                notable differencesbull Now try to improve ldquophase classificationrdquo

                                                Yeast Cell CycleRevisit ldquophase classificationrdquo

                                                approachbull Use outer 200 genes

                                                (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                                (finds significant bumps etc in histogram)

                                                bull Carefully redrew boundariesbull Check by studying kde angles

                                                SiZer Study of Distrsquon of Angles

                                                Reclassification of Major Genes

                                                Compare to Previous Classifrsquon

                                                New Subpopulation View

                                                New Subpopulation View

                                                NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                Detailed Look at PCA

                                                Now Study ldquoFolklorerdquo More Carefully

                                                bull BackGround

                                                bull History

                                                bull Underpinnings

                                                (Mathematical amp Computational)

                                                Good Overall Reference Jolliffe (2002)

                                                PCA Rediscovery ndash Renaming

                                                Statistics Principal Component Analysis (PCA)

                                                PCA Rediscovery ndash Renaming

                                                Statistics Principal Component Analysis (PCA)

                                                Social Sciences Factor Analysis (PCA is a subset)

                                                PCA Rediscovery ndash Renaming

                                                Statistics Principal Component Analysis (PCA)

                                                Social Sciences Factor Analysis (PCA is a subset)

                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                PCA Rediscovery ndash Renaming

                                                Statistics Principal Component Analysis (PCA)

                                                Social Sciences Factor Analysis (PCA is a subset)

                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                                PCA Rediscovery ndash Renaming

                                                Statistics Principal Component Analysis (PCA)

                                                Social Sciences Factor Analysis (PCA is a subset)

                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                                Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                An Interesting Historical Note

                                                The 1st () application of PCA to Functional

                                                Data Analysis

                                                An Interesting Historical Note

                                                The 1st () application of PCA to Functional

                                                Data Analysis

                                                Rao (1958)

                                                1st Paper with ldquoCurves as Data Objectsrdquo

                                                viewpoint

                                                Detailed Look at PCA

                                                Three Important (amp Interesting) Viewpoints

                                                1 Mathematics

                                                2 Numerics

                                                3 Statistics

                                                Goal Study Interrelationships

                                                Detailed Look at PCA

                                                Three Important (amp Interesting) Viewpoints

                                                1 Mathematics

                                                2 Numerics

                                                3 Statistics

                                                1st Review Linear Alg and Multivar Prob

                                                Review of Linear Algebra

                                                Vector Space

                                                bull set of ldquovectorsrdquo

                                                bull and ldquoscalarsrdquo (coefficients)

                                                xa

                                                Review of Linear Algebra

                                                Vector Space

                                                bull set of ldquovectorsrdquo

                                                bull and ldquoscalarsrdquo (coefficients)

                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                ( in space)

                                                xa

                                                i

                                                ii xa

                                                Review of Linear Algebra

                                                Vector Space

                                                bull set of ldquovectorsrdquo

                                                bull and ldquoscalarsrdquo (coefficients)

                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                ( in space)

                                                eg

                                                ldquo dim Euclidrsquon spacerdquo

                                                xa

                                                i

                                                ii xa

                                                d

                                                d

                                                d xx

                                                x

                                                x

                                                x 1

                                                1

                                                d

                                                Review of Linear Algebra (Cont)

                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                Review of Linear Algebra (Cont)

                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                Review of Linear Algebra (Cont)

                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                Note Planes not Through the Origin

                                                are not Subspaces

                                                (Do not Contain )00 x

                                                Review of Linear Algebra (Cont)

                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                (all Linear Combos of them =

                                                = Containing Hyperplane

                                                through Origin)

                                                Review of Linear Algebra (Cont)

                                                Basis of Subspace Set of Vectors that

                                                bull Span ie Everything is a Lin Com of them

                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                Review of Linear Algebra (Cont)

                                                Basis of Subspace Set of Vectors that

                                                bull Span ie Everything is a Lin Com of them

                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                bull eg ldquoUnit Vector Basisrdquo

                                                bull Since

                                                d

                                                1

                                                0

                                                0

                                                0

                                                1

                                                0

                                                0

                                                0

                                                1

                                                1

                                                0

                                                0

                                                0

                                                1

                                                0

                                                0

                                                0

                                                1

                                                212

                                                1

                                                d

                                                d

                                                xxx

                                                x

                                                x

                                                x

                                                Review of Linear Algebra (Cont)

                                                Basis Matrix of subspace of

                                                Given a basis

                                                create matrix of columns

                                                dnvv 1

                                                nddnd

                                                n

                                                n

                                                vv

                                                vv

                                                vvB

                                                1

                                                111

                                                1

                                                Review of Linear Algebra (Cont)

                                                Then linear combo is a matrix multiplicatrsquon

                                                where

                                                n

                                                iii aBva

                                                1

                                                na

                                                a

                                                a 1

                                                Review of Linear Algebra (Cont)

                                                Then linear combo is a matrix multiplicatrsquon

                                                where

                                                Note Right Multiplication Gives

                                                Linear Combination of Column Vectors

                                                n

                                                iii aBva

                                                1

                                                na

                                                a

                                                a 1

                                                Review of Linear Algebra (Cont)

                                                Then linear combo is a matrix multiplicatrsquon

                                                where

                                                Check sizes

                                                n

                                                iii aBva

                                                1

                                                na

                                                a

                                                a 1

                                                )1()(1 nndd

                                                Review of Linear Algebra (Cont)

                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                For matrices

                                                Define the Matrix Product

                                                mkk

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                nmm

                                                n

                                                bb

                                                bb

                                                B

                                                1

                                                111

                                                m

                                                iniik

                                                m

                                                iiik

                                                m

                                                inii

                                                m

                                                iii

                                                baba

                                                baba

                                                AB

                                                1

                                                11

                                                11

                                                111

                                                Review of Linear Algebra (Cont)

                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                For matrices

                                                Define the Matrix Product

                                                (Inner Products of Rows With Columns )

                                                mkk

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                nmm

                                                n

                                                bb

                                                bb

                                                B

                                                1

                                                111

                                                m

                                                iniik

                                                m

                                                iiik

                                                m

                                                inii

                                                m

                                                iii

                                                baba

                                                baba

                                                AB

                                                1

                                                11

                                                11

                                                111

                                                A B

                                                Review of Linear Algebra (Cont)

                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                For matrices

                                                Define the Matrix Product

                                                (Inner Products of Rows With Columns )

                                                (Composition of Linear Transformations)

                                                mkk

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                nmm

                                                n

                                                bb

                                                bb

                                                B

                                                1

                                                111

                                                m

                                                iniik

                                                m

                                                iiik

                                                m

                                                inii

                                                m

                                                iii

                                                baba

                                                baba

                                                AB

                                                1

                                                11

                                                11

                                                111

                                                A B

                                                Review of Linear Algebra (Cont)

                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                For matrices

                                                Define the Matrix Product

                                                (Inner Products of Rows With Columns )

                                                (Composition of Linear Transformations)

                                                Often Useful to Check Sizes

                                                mkk

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                nmm

                                                n

                                                bb

                                                bb

                                                B

                                                1

                                                111

                                                m

                                                iniik

                                                m

                                                iiik

                                                m

                                                inii

                                                m

                                                iii

                                                baba

                                                baba

                                                AB

                                                1

                                                11

                                                11

                                                111

                                                nmmknk

                                                A B

                                                Review of Linear Algebra (Cont)

                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                For matrices

                                                Define the Matrix Product

                                                (Inner Products of Rows With Columns )

                                                (Composition of Linear Transformations)

                                                Often Useful to Check Sizes

                                                mkk

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                nmm

                                                n

                                                bb

                                                bb

                                                B

                                                1

                                                111

                                                m

                                                iniik

                                                m

                                                iiik

                                                m

                                                inii

                                                m

                                                iii

                                                baba

                                                baba

                                                AB

                                                1

                                                11

                                                11

                                                111

                                                nmmknk

                                                A B

                                                Review of Linear Algebra (Cont)

                                                Matrix Trace

                                                bull For a Square Matrix

                                                bull Define

                                                mmm

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                m

                                                iiiaAtr

                                                1)(

                                                Review of Linear Algebra (Cont)

                                                Matrix Trace

                                                bull For a Square Matrix

                                                bull Define

                                                bull Trace Commutes with Matrix Multiplication

                                                mmm

                                                m

                                                aa

                                                aa

                                                A

                                                1

                                                111

                                                m

                                                iiiaAtr

                                                1)(

                                                BAtrABtr

                                                Review of Linear Algebra (Cont)

                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                bull Number of Elements in a Basis (Unique)

                                                Review of Linear Algebra (Cont)

                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                bull Number of Elements in a Basis (Unique)

                                                bull (Use Basis Above)

                                                bull eg dim of a line is 1

                                                bull eg dim of a plane is 2

                                                dd dim

                                                Review of Linear Algebra (Cont)

                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                bull Number of Elements in a Basis (Unique)

                                                bull (Use Basis Above)

                                                bull eg dim of a line is 1

                                                bull eg dim of a plane is 2

                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                (in Statistical Uses eg ANOVA)

                                                dd dim

                                                Review of Linear Algebra (Cont)

                                                Norm of a Vector

                                                bull in d 21

                                                21

                                                1

                                                2 xxxx td

                                                jj

                                                Review of Linear Algebra (Cont)

                                                Norm of a Vector

                                                bull in

                                                bull Idea length of the vector

                                                d 2121

                                                1

                                                2 xxxx td

                                                jj

                                                Review of Linear Algebra (Cont)

                                                Norm of a Vector

                                                bull in

                                                bull Idea length of the vector

                                                bull Note strange properties for high

                                                eg ldquolength of diagonal of unit cuberdquo =

                                                d 2121

                                                1

                                                2 xxxx td

                                                jj

                                                d

                                                d

                                                Review of Linear Algebra (Cont)

                                                Norm of a Vector (cont)

                                                bull Length Normalized Vector

                                                (has Length 1 thus on Surf of Unit Sphere

                                                amp is a Direction Vector)

                                                x

                                                x

                                                Review of Linear Algebra (Cont)

                                                Norm of a Vector (cont)

                                                bull Length Normalized Vector

                                                (has Length 1 thus on Surf of Unit Sphere

                                                amp is a Direction Vector)

                                                bull Define Distance as

                                                x

                                                x

                                                yxyxyxyxd t

                                                Review of Linear Algebra (Cont)

                                                Inner (Dot Scalar) Product

                                                bull for Vectors and

                                                yxyxyx td

                                                jjj

                                                1

                                                x y

                                                Review of Linear Algebra (Cont)

                                                Inner (Dot Scalar) Product

                                                bull for Vectors and

                                                bull Related to Norm via

                                                yxyxyx td

                                                jjj

                                                1

                                                x y

                                                21 xxx

                                                Review of Linear Algebra (Cont)

                                                Inner (Dot Scalar) Product (cont)

                                                bull measures ldquoangle between and rdquo as

                                                yyxx

                                                yx

                                                yx

                                                yxyxangle

                                                tt

                                                t

                                                11 cos

                                                cos

                                                x y

                                                Review of Linear Algebra (Cont)

                                                Inner (Dot Scalar) Product (cont)

                                                bull measures ldquoangle between and rdquo as

                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                if and only if

                                                yyxx

                                                yx

                                                yx

                                                yxyxangle

                                                tt

                                                t

                                                11 cos

                                                cos

                                                x y

                                                yx 0 yx

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis

                                                bull All Orthogonal to each other

                                                ie for

                                                bull All have Length 1

                                                ie for

                                                nvv 1

                                                1 ii vv

                                                0 ii vv ii

                                                ni 1

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis (cont)

                                                bull Spectral Representation

                                                where

                                                nvv 1

                                                n

                                                iii vax

                                                1

                                                ii vxa

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis (cont)

                                                bull Spectral Representation

                                                where

                                                (Coefficient is Inner Product Cool Notation)

                                                nvv 1

                                                n

                                                iii vax

                                                1

                                                ii vxa

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis (cont)

                                                bull Spectral Representation

                                                where

                                                Check

                                                nvv 1

                                                n

                                                iii vax

                                                1

                                                ii vxa

                                                iii

                                                n

                                                iii

                                                n

                                                iiii avvavvavx

                                                1

                                                1

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis (cont)

                                                bull Spectral Representation

                                                where

                                                Check

                                                bull Matrix Notation where ie

                                                For the Basis Matrix

                                                nvv 1

                                                n

                                                iii vax

                                                1

                                                ii vxa

                                                iii

                                                n

                                                iii

                                                n

                                                iiii avvavvavx

                                                1

                                                1

                                                aBx Bxa tt xBa t

                                                nvvB 1

                                                Review of Linear Algebra (Cont)

                                                Orthonormal Basis (cont)

                                                bull Spectral Representation

                                                where

                                                Check

                                                bull Matrix Notation where ie

                                                is called transform of

                                                (eg Fourier or Wavelet)

                                                nvv 1

                                                n

                                                iii vax

                                                1

                                                ii vxa

                                                iii

                                                n

                                                iii

                                                n

                                                iiii avvavvavx

                                                1

                                                1

                                                aBx Bxa tt xBa t

                                                xa

                                                Review of Linear Algebra (Cont)

                                                Parseval identity for

                                                in subsp genrsquod by o n basis

                                                x

                                                nvv 1

                                                2

                                                1

                                                22

                                                1

                                                2 aavxx

                                                n

                                                ii

                                                n

                                                ii

                                                Review of Linear Algebra (Cont)

                                                Parseval identity for

                                                in subsp genrsquod by o n basis

                                                bull Pythagorean theorem

                                                bull ldquoDecomposition of Energyrdquo

                                                bull ANOVA - sums of squares

                                                x

                                                nvv 1

                                                2

                                                1

                                                22

                                                1

                                                2 aavxx

                                                n

                                                ii

                                                n

                                                ii

                                                Review of Linear Algebra (Cont)

                                                Parseval identity for

                                                in subsp genrsquod by o n basis

                                                bull Pythagorean theorem

                                                bull ldquoDecomposition of Energyrdquo

                                                bull ANOVA - sums of squares

                                                bull Transform has same length as

                                                ie ldquorotation in rdquo

                                                x

                                                nvv 1

                                                2

                                                1

                                                22

                                                1

                                                2 aavxx

                                                n

                                                ii

                                                n

                                                ii

                                                a xd

                                                Projection of a Vector onto a Subspace

                                                bull Idea Member of that is Closest to

                                                (ie ldquoBest Approxrsquonrdquo)

                                                Review of Linear Algebra (Cont)x

                                                xV

                                                V

                                                Projection of a Vector onto a Subspace

                                                bull Idea Member of that is Closest to

                                                (ie ldquoBest Approxrsquonrdquo)

                                                bull Find that Solves

                                                (ldquoLeast Squaresrdquo)

                                                Review of Linear Algebra (Cont)x

                                                xV

                                                V

                                                VxPV vxVv

                                                min

                                                Projection of a Vector onto a Subspace

                                                bull Idea Member of that is Closest to

                                                (ie ldquoBest Approxrsquonrdquo)

                                                bull Find that Solves

                                                (ldquoLeast Squaresrdquo)

                                                bull For Inner Product (Hilbert) Space

                                                Exists and is Unique

                                                Review of Linear Algebra (Cont)x

                                                xV

                                                V

                                                VxPV vxVv

                                                min

                                                xPV

                                                Projection of a Vector onto a Subspace (cont)

                                                bull General Solution in for Basis Matrix

                                                Review of Linear Algebra (Cont)

                                                d VB

                                                xBBBBxP tVV

                                                tVVV

                                                1

                                                Projection of a Vector onto a Subspace (cont)

                                                bull General Solution in for Basis Matrix

                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                (thus projection is another linear operation)

                                                Review of Linear Algebra (Cont)

                                                d VB

                                                xBBBBxP tVV

                                                tVVV

                                                1

                                                tVV

                                                tVVV BBBBP

                                                1

                                                Projection of a Vector onto a Subspace (cont)

                                                bull General Solution in for Basis Matrix

                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                (thus projection is another linear operation)

                                                (note same operation underlies least squares)

                                                Review of Linear Algebra (Cont)

                                                d VB

                                                xBBBBxP tVV

                                                tVVV

                                                1

                                                tVV

                                                tVVV BBBBP

                                                1

                                                Review of Linear Algebra (Cont)

                                                Projection using Orthonormal Basis

                                                bull Basis Matrix is Orthonormal

                                                nnVtV IBB

                                                10

                                                01

                                                1

                                                111

                                                1

                                                1

                                                nnn

                                                n

                                                ntn

                                                t

                                                vvvv

                                                vvvv

                                                vv

                                                v

                                                v

                                                nvv 1

                                                Review of Linear Algebra (Cont)

                                                Projection using Orthonormal Basis

                                                bull Basis Matrix is Orthonormal

                                                bull So =

                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                (Recall Right Multrsquon)

                                                nnVtV IBB

                                                10

                                                01

                                                1

                                                111

                                                1

                                                1

                                                nnn

                                                n

                                                ntn

                                                t

                                                vvvv

                                                vvvv

                                                vv

                                                v

                                                v

                                                xBBxP tVVV

                                                x V

                                                nvv 1

                                                Review of Linear Algebra (Cont)

                                                Projection using Orthonormal Basis (cont)

                                                bull For Orthogonal Complement

                                                and

                                                V

                                                xPxPx VV 222xPxPx VV

                                                Review of Linear Algebra (Cont)

                                                Projection using Orthonormal Basis (cont)

                                                bull For Orthogonal Complement

                                                and

                                                bull Parseval Inequality

                                                V

                                                xPxPx VV 222xPxPx VV

                                                2

                                                1

                                                22

                                                1

                                                22 aavxxxP

                                                n

                                                ii

                                                n

                                                iiV

                                                Review of Linear Algebra (Cont)

                                                (Real) Unitary Matrices with

                                                bull Orthonormal Basis Matrix

                                                (So All of Above Applies)

                                                ddU IUU t

                                                Review of Linear Algebra (Cont)

                                                (Real) Unitary Matrices with

                                                bull Orthonormal Basis Matrix

                                                (So All of Above Applies)

                                                bull Note Transformrsquon is Distance Preserving

                                                ddU IUU t

                                                yxdyxyxyxUyUxUdn

                                                i ii 2

                                                1

                                                Review of Linear Algebra (Cont)

                                                (Real) Unitary Matrices with

                                                bull Orthonormal Basis Matrix

                                                (So All of Above Applies)

                                                bull Note Transformrsquon is Distance Preserving

                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                bull But also Includes ldquoMirror Imagesrdquo

                                                ddU IUU t

                                                yxdyxyxyxUyUxUdn

                                                i ii 2

                                                1

                                                U

                                                Review of Linear Algebra (Cont)

                                                Singular Value Decomposition (SVD)

                                                For a Matrix

                                                Find

                                                ndX

                                                Review of Linear Algebra (Cont)

                                                Singular Value Decomposition (SVD)

                                                For a Matrix

                                                Find a Diagonal Matrix

                                                with Entries

                                                called Singular Values

                                                ndX

                                                ndS

                                                )min(1 ndss

                                                Review of Linear Algebra (Cont)

                                                Singular Value Decomposition (SVD)

                                                For a Matrix

                                                Find a Diagonal Matrix

                                                with Entries

                                                called Singular Values

                                                And Unitary (Rotation) Matrices

                                                (recall )

                                                ndX

                                                ndS

                                                )min(1 ndss

                                                ddU nnV

                                                IVVUU tt

                                                Review of Linear Algebra (Cont)

                                                Singular Value Decomposition (SVD)

                                                For a Matrix

                                                Find a Diagonal Matrix

                                                with Entries

                                                called Singular Values

                                                And Unitary (Rotation) Matrices

                                                (recall )

                                                So That

                                                ndX

                                                ndS

                                                )min(1 ndss

                                                ddU nnV

                                                IVVUU tt tUSVX

                                                Review of Linear Algebra (Cont)

                                                Intuition behind Singular Value Decomposition

                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                vVSUvVSUvX tt

                                                Review of Linear Algebra (Cont)

                                                Intuition behind Singular Value Decomposition

                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                bull First rotate

                                                X

                                                vVSUvVSUvX tt

                                                Review of Linear Algebra (Cont)

                                                Intuition behind Singular Value Decomposition

                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                bull First rotate

                                                bull Second rescale coordinate axes (by )

                                                X

                                                vVSUvVSUvX tt

                                                is

                                                Review of Linear Algebra (Cont)

                                                Intuition behind Singular Value Decomposition

                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                bull First rotate

                                                bull Second rescale coordinate axes (by )

                                                bull Third rotate again

                                                X

                                                vVSUvVSUvX tt

                                                is

                                                Review of Linear Algebra (Cont)

                                                Intuition behind Singular Value Decomposition

                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                bull First rotate

                                                bull Second rescale coordinate axes (by )

                                                bull Third rotate again

                                                bull ie have diagonalized the transformation

                                                X

                                                vVSUvVSUvX tt

                                                is

                                                SVD Compact Representation

                                                Useful Labeling

                                                Singular Values in Increasing Order

                                                Review of Linear Algebra (Cont)

                                                )min(1 dnss

                                                SVD Compact Representation

                                                Useful Labeling

                                                Singular Values in Increasing Order

                                                Note singular values = 0 can be omitted

                                                (Since do ldquo0-Stretchingrdquo)

                                                Review of Linear Algebra (Cont)

                                                )min(1 dnss

                                                Review of Linear Algebra (Cont)

                                                r

                                                SVD Compact Representation

                                                Useful Labeling

                                                Singular Values in Increasing Order

                                                Note singular values = 0 can be omitted

                                                Let = of positive singular values

                                                )min(1 dnss

                                                Review of Linear Algebra (Cont)

                                                r

                                                SVD Compact Representation

                                                Useful Labeling

                                                Singular Values in Increasing Order

                                                Note singular values = 0 can be omitted

                                                Let = of positive singular values

                                                Then

                                                Where are truncations of

                                                trnrrrd VSUX

                                                VSU

                                                )min(1 dnss

                                                Review of Linear Algebra (Cont)

                                                SVD Full Representation

                                                =

                                                Graphics Display Assumes

                                                ndX ddU ndS nn

                                                tV

                                                nd

                                                Review of Linear Algebra (Cont)

                                                SVD Full Representation

                                                =

                                                Full Rank Basis Matrix

                                                ndX ddU ndS nn

                                                tV

                                                Review of Linear Algebra (Cont)

                                                SVD Full Representation

                                                =

                                                Full Rank Basis Matrix

                                                All 0s in Bottom

                                                ndX ddU ndS nn

                                                tV

                                                Review of Linear Algebra (Cont)

                                                SVD Reduced Representation

                                                =

                                                These Columns Get 0ed Out

                                                ndX ddU nnS nn

                                                tV

                                                nnd 0

                                                Review of Linear Algebra (Cont)

                                                SVD Reduced Representation

                                                =ndX ndU nnS nn

                                                tV

                                                Review of Linear Algebra (Cont)

                                                SVD Reduced Representation

                                                =

                                                Also Some of These May be 0

                                                ndX ndU nnS nn

                                                tV

                                                Review of Linear Algebra (Cont)

                                                SVD Compact Representation

                                                = ndX rdU

                                                rrS nrtV

                                                0

                                                Review of Linear Algebra (Cont)

                                                SVD Compact Representation

                                                =

                                                These Get 0ed Out

                                                ndX rdU

                                                rrS nrtV

                                                0

                                                Review of Linear Algebra (Cont)

                                                SVD Compact Representation

                                                = ndX rdU

                                                rrS nrtV

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition

                                                For a (Symmetric) Square Matrix

                                                Find

                                                ddX

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition

                                                For a (Symmetric) Square Matrix

                                                Find a Diagonal Matrix

                                                Called Eigenvalues

                                                Convenient Ordering

                                                ddX

                                                d

                                                D

                                                0

                                                01

                                                n 1

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition

                                                For a (Symmetric) Square Matrix

                                                Find a Diagonal Matrix

                                                And an Orthonormal Matrix

                                                (ie )

                                                ddX

                                                d

                                                D

                                                0

                                                01

                                                ddB

                                                ddtt IBBBB

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition

                                                For a (Symmetric) Square Matrix

                                                Find a Diagonal Matrix

                                                And an Orthonormal Matrix

                                                (ie )

                                                So that ie

                                                ddX

                                                d

                                                D

                                                0

                                                01

                                                ddB

                                                ddtt IBBBB

                                                DBBX tBDBX

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                (looks similar)

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                Complex (uses )

                                                VU

                                                1i

                                                Review of Linear Algebra (Cont)

                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                VU

                                                X

                                                BVU

                                                1i

                                                • Participant Presentations
                                                • PCA to find clusters
                                                • Statistical Smoothing
                                                • Kernel Density Estimation
                                                • Scatterplot Smoothing
                                                • Statistical Smoothing (2)
                                                • SiZer Background
                                                • SiZer Background (2)
                                                • SiZer Background (3)
                                                • SiZer Background (4)
                                                • SiZer Background (5)
                                                • SiZer Overview
                                                • PCA to find clusters (2)
                                                • PCA to find clusters (3)
                                                • PCA to find clusters (4)
                                                • PCA to find clusters (5)
                                                • PCA to find clusters (6)
                                                • PCA to find clusters (7)
                                                • Recall Yeast Cell Cycle Data
                                                • Yeast Cell Cycle Data FDA View
                                                • Yeast Cell Cycle Data FDA View (2)
                                                • Yeast Cell Cycle Data FDA View (3)
                                                • Yeast Cell Cycles Freq 2 Proj
                                                • Frequency 2 Analysis
                                                • Frequency 2 Analysis (2)
                                                • Frequency 2 Analysis (3)
                                                • Yeast Cell Cycle
                                                • SiZer Study of Distrsquon of Angles
                                                • Reclassification of Major Genes
                                                • Compare to Previous Classifrsquon
                                                • New Subpopulation View
                                                • New Subpopulation View (2)
                                                • Detailed Look at PCA
                                                • PCA Rediscovery ndash Renaming
                                                • PCA Rediscovery ndash Renaming (2)
                                                • PCA Rediscovery ndash Renaming (3)
                                                • PCA Rediscovery ndash Renaming (4)
                                                • PCA Rediscovery ndash Renaming (5)
                                                • An Interesting Historical Note
                                                • An Interesting Historical Note (2)
                                                • Detailed Look at PCA (2)
                                                • Detailed Look at PCA (3)
                                                • Review of Linear Algebra
                                                • Review of Linear Algebra (2)
                                                • Review of Linear Algebra (3)
                                                • Review of Linear Algebra (Cont)
                                                • Review of Linear Algebra (Cont) (2)
                                                • Review of Linear Algebra (Cont) (3)
                                                • Review of Linear Algebra (Cont) (4)
                                                • Review of Linear Algebra (Cont) (5)
                                                • Review of Linear Algebra (Cont) (6)
                                                • Review of Linear Algebra (Cont) (7)
                                                • Review of Linear Algebra (Cont) (8)
                                                • Review of Linear Algebra (Cont) (9)
                                                • Review of Linear Algebra (Cont) (10)
                                                • Review of Linear Algebra (Cont) (11)
                                                • Review of Linear Algebra (Cont) (12)
                                                • Review of Linear Algebra (Cont) (13)
                                                • Review of Linear Algebra (Cont) (14)
                                                • Review of Linear Algebra (Cont) (15)
                                                • Review of Linear Algebra (Cont) (16)
                                                • Review of Linear Algebra (Cont) (17)
                                                • Review of Linear Algebra (Cont) (18)
                                                • Review of Linear Algebra (Cont) (19)
                                                • Review of Linear Algebra (Cont) (20)
                                                • Review of Linear Algebra (Cont) (21)
                                                • Review of Linear Algebra (Cont) (22)
                                                • Review of Linear Algebra (Cont) (23)
                                                • Review of Linear Algebra (Cont) (24)
                                                • Review of Linear Algebra (Cont) (25)
                                                • Review of Linear Algebra (Cont) (26)
                                                • Review of Linear Algebra (Cont) (27)
                                                • Review of Linear Algebra (Cont) (28)
                                                • Review of Linear Algebra (Cont) (29)
                                                • Review of Linear Algebra (Cont) (30)
                                                • Review of Linear Algebra (Cont) (31)
                                                • Review of Linear Algebra (Cont) (32)
                                                • Review of Linear Algebra (Cont) (33)
                                                • Review of Linear Algebra (Cont) (34)
                                                • Review of Linear Algebra (Cont) (35)
                                                • Review of Linear Algebra (Cont) (36)
                                                • Review of Linear Algebra (Cont) (37)
                                                • Review of Linear Algebra (Cont) (38)
                                                • Review of Linear Algebra (Cont) (39)
                                                • Review of Linear Algebra (Cont) (40)
                                                • Review of Linear Algebra (Cont) (41)
                                                • Review of Linear Algebra (Cont) (42)
                                                • Review of Linear Algebra (Cont) (43)
                                                • Review of Linear Algebra (Cont) (44)
                                                • Review of Linear Algebra (Cont) (45)
                                                • Review of Linear Algebra (Cont) (46)
                                                • Review of Linear Algebra (Cont) (47)
                                                • Review of Linear Algebra (Cont) (48)
                                                • Review of Linear Algebra (Cont) (49)
                                                • Review of Linear Algebra (Cont) (50)
                                                • Review of Linear Algebra (Cont) (51)
                                                • Review of Linear Algebra (Cont) (52)
                                                • Review of Linear Algebra (Cont) (53)
                                                • Review of Linear Algebra (Cont) (54)
                                                • Review of Linear Algebra (Cont) (55)
                                                • Review of Linear Algebra (Cont) (56)
                                                • Review of Linear Algebra (Cont) (57)
                                                • Review of Linear Algebra (Cont) (58)
                                                • Review of Linear Algebra (Cont) (59)
                                                • Review of Linear Algebra (Cont) (60)
                                                • Review of Linear Algebra (Cont) (61)
                                                • Review of Linear Algebra (Cont) (62)
                                                • Review of Linear Algebra (Cont) (63)
                                                • Review of Linear Algebra (Cont) (64)
                                                • Review of Linear Algebra (Cont) (65)
                                                • Review of Linear Algebra (Cont) (66)
                                                • Review of Linear Algebra (Cont) (67)
                                                • Review of Linear Algebra (Cont) (68)
                                                • Review of Linear Algebra (Cont) (69)
                                                • Review of Linear Algebra (Cont) (70)
                                                • Review of Linear Algebra (Cont) (71)
                                                • Review of Linear Algebra (Cont) (72)
                                                • Review of Linear Algebra (Cont) (73)
                                                • Review of Linear Algebra (Cont) (74)
                                                • Review of Linear Algebra (Cont) (75)
                                                • Review of Linear Algebra (Cont) (76)
                                                • Review of Linear Algebra (Cont) (77)
                                                • Review of Linear Algebra (Cont) (78)
                                                • Review of Linear Algebra (Cont) (79)
                                                • Review of Linear Algebra (Cont) (80)
                                                • Review of Linear Algebra (Cont) (81)

                                                  Frequency 2 Analysis

                                                  Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                                  cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                                  classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                                  notable differencesbull Now try to improve ldquophase classificationrdquo

                                                  Yeast Cell CycleRevisit ldquophase classificationrdquo

                                                  approachbull Use outer 200 genes

                                                  (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                                  (finds significant bumps etc in histogram)

                                                  bull Carefully redrew boundariesbull Check by studying kde angles

                                                  SiZer Study of Distrsquon of Angles

                                                  Reclassification of Major Genes

                                                  Compare to Previous Classifrsquon

                                                  New Subpopulation View

                                                  New Subpopulation View

                                                  NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                  Detailed Look at PCA

                                                  Now Study ldquoFolklorerdquo More Carefully

                                                  bull BackGround

                                                  bull History

                                                  bull Underpinnings

                                                  (Mathematical amp Computational)

                                                  Good Overall Reference Jolliffe (2002)

                                                  PCA Rediscovery ndash Renaming

                                                  Statistics Principal Component Analysis (PCA)

                                                  PCA Rediscovery ndash Renaming

                                                  Statistics Principal Component Analysis (PCA)

                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                  PCA Rediscovery ndash Renaming

                                                  Statistics Principal Component Analysis (PCA)

                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                  PCA Rediscovery ndash Renaming

                                                  Statistics Principal Component Analysis (PCA)

                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                                  PCA Rediscovery ndash Renaming

                                                  Statistics Principal Component Analysis (PCA)

                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                                  Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                  An Interesting Historical Note

                                                  The 1st () application of PCA to Functional

                                                  Data Analysis

                                                  An Interesting Historical Note

                                                  The 1st () application of PCA to Functional

                                                  Data Analysis

                                                  Rao (1958)

                                                  1st Paper with ldquoCurves as Data Objectsrdquo

                                                  viewpoint

                                                  Detailed Look at PCA

                                                  Three Important (amp Interesting) Viewpoints

                                                  1 Mathematics

                                                  2 Numerics

                                                  3 Statistics

                                                  Goal Study Interrelationships

                                                  Detailed Look at PCA

                                                  Three Important (amp Interesting) Viewpoints

                                                  1 Mathematics

                                                  2 Numerics

                                                  3 Statistics

                                                  1st Review Linear Alg and Multivar Prob

                                                  Review of Linear Algebra

                                                  Vector Space

                                                  bull set of ldquovectorsrdquo

                                                  bull and ldquoscalarsrdquo (coefficients)

                                                  xa

                                                  Review of Linear Algebra

                                                  Vector Space

                                                  bull set of ldquovectorsrdquo

                                                  bull and ldquoscalarsrdquo (coefficients)

                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                  ( in space)

                                                  xa

                                                  i

                                                  ii xa

                                                  Review of Linear Algebra

                                                  Vector Space

                                                  bull set of ldquovectorsrdquo

                                                  bull and ldquoscalarsrdquo (coefficients)

                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                  ( in space)

                                                  eg

                                                  ldquo dim Euclidrsquon spacerdquo

                                                  xa

                                                  i

                                                  ii xa

                                                  d

                                                  d

                                                  d xx

                                                  x

                                                  x

                                                  x 1

                                                  1

                                                  d

                                                  Review of Linear Algebra (Cont)

                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                  Review of Linear Algebra (Cont)

                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                  Review of Linear Algebra (Cont)

                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                  Note Planes not Through the Origin

                                                  are not Subspaces

                                                  (Do not Contain )00 x

                                                  Review of Linear Algebra (Cont)

                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                  (all Linear Combos of them =

                                                  = Containing Hyperplane

                                                  through Origin)

                                                  Review of Linear Algebra (Cont)

                                                  Basis of Subspace Set of Vectors that

                                                  bull Span ie Everything is a Lin Com of them

                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                  Review of Linear Algebra (Cont)

                                                  Basis of Subspace Set of Vectors that

                                                  bull Span ie Everything is a Lin Com of them

                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                  bull eg ldquoUnit Vector Basisrdquo

                                                  bull Since

                                                  d

                                                  1

                                                  0

                                                  0

                                                  0

                                                  1

                                                  0

                                                  0

                                                  0

                                                  1

                                                  1

                                                  0

                                                  0

                                                  0

                                                  1

                                                  0

                                                  0

                                                  0

                                                  1

                                                  212

                                                  1

                                                  d

                                                  d

                                                  xxx

                                                  x

                                                  x

                                                  x

                                                  Review of Linear Algebra (Cont)

                                                  Basis Matrix of subspace of

                                                  Given a basis

                                                  create matrix of columns

                                                  dnvv 1

                                                  nddnd

                                                  n

                                                  n

                                                  vv

                                                  vv

                                                  vvB

                                                  1

                                                  111

                                                  1

                                                  Review of Linear Algebra (Cont)

                                                  Then linear combo is a matrix multiplicatrsquon

                                                  where

                                                  n

                                                  iii aBva

                                                  1

                                                  na

                                                  a

                                                  a 1

                                                  Review of Linear Algebra (Cont)

                                                  Then linear combo is a matrix multiplicatrsquon

                                                  where

                                                  Note Right Multiplication Gives

                                                  Linear Combination of Column Vectors

                                                  n

                                                  iii aBva

                                                  1

                                                  na

                                                  a

                                                  a 1

                                                  Review of Linear Algebra (Cont)

                                                  Then linear combo is a matrix multiplicatrsquon

                                                  where

                                                  Check sizes

                                                  n

                                                  iii aBva

                                                  1

                                                  na

                                                  a

                                                  a 1

                                                  )1()(1 nndd

                                                  Review of Linear Algebra (Cont)

                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                  For matrices

                                                  Define the Matrix Product

                                                  mkk

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  nmm

                                                  n

                                                  bb

                                                  bb

                                                  B

                                                  1

                                                  111

                                                  m

                                                  iniik

                                                  m

                                                  iiik

                                                  m

                                                  inii

                                                  m

                                                  iii

                                                  baba

                                                  baba

                                                  AB

                                                  1

                                                  11

                                                  11

                                                  111

                                                  Review of Linear Algebra (Cont)

                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                  For matrices

                                                  Define the Matrix Product

                                                  (Inner Products of Rows With Columns )

                                                  mkk

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  nmm

                                                  n

                                                  bb

                                                  bb

                                                  B

                                                  1

                                                  111

                                                  m

                                                  iniik

                                                  m

                                                  iiik

                                                  m

                                                  inii

                                                  m

                                                  iii

                                                  baba

                                                  baba

                                                  AB

                                                  1

                                                  11

                                                  11

                                                  111

                                                  A B

                                                  Review of Linear Algebra (Cont)

                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                  For matrices

                                                  Define the Matrix Product

                                                  (Inner Products of Rows With Columns )

                                                  (Composition of Linear Transformations)

                                                  mkk

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  nmm

                                                  n

                                                  bb

                                                  bb

                                                  B

                                                  1

                                                  111

                                                  m

                                                  iniik

                                                  m

                                                  iiik

                                                  m

                                                  inii

                                                  m

                                                  iii

                                                  baba

                                                  baba

                                                  AB

                                                  1

                                                  11

                                                  11

                                                  111

                                                  A B

                                                  Review of Linear Algebra (Cont)

                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                  For matrices

                                                  Define the Matrix Product

                                                  (Inner Products of Rows With Columns )

                                                  (Composition of Linear Transformations)

                                                  Often Useful to Check Sizes

                                                  mkk

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  nmm

                                                  n

                                                  bb

                                                  bb

                                                  B

                                                  1

                                                  111

                                                  m

                                                  iniik

                                                  m

                                                  iiik

                                                  m

                                                  inii

                                                  m

                                                  iii

                                                  baba

                                                  baba

                                                  AB

                                                  1

                                                  11

                                                  11

                                                  111

                                                  nmmknk

                                                  A B

                                                  Review of Linear Algebra (Cont)

                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                  For matrices

                                                  Define the Matrix Product

                                                  (Inner Products of Rows With Columns )

                                                  (Composition of Linear Transformations)

                                                  Often Useful to Check Sizes

                                                  mkk

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  nmm

                                                  n

                                                  bb

                                                  bb

                                                  B

                                                  1

                                                  111

                                                  m

                                                  iniik

                                                  m

                                                  iiik

                                                  m

                                                  inii

                                                  m

                                                  iii

                                                  baba

                                                  baba

                                                  AB

                                                  1

                                                  11

                                                  11

                                                  111

                                                  nmmknk

                                                  A B

                                                  Review of Linear Algebra (Cont)

                                                  Matrix Trace

                                                  bull For a Square Matrix

                                                  bull Define

                                                  mmm

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  m

                                                  iiiaAtr

                                                  1)(

                                                  Review of Linear Algebra (Cont)

                                                  Matrix Trace

                                                  bull For a Square Matrix

                                                  bull Define

                                                  bull Trace Commutes with Matrix Multiplication

                                                  mmm

                                                  m

                                                  aa

                                                  aa

                                                  A

                                                  1

                                                  111

                                                  m

                                                  iiiaAtr

                                                  1)(

                                                  BAtrABtr

                                                  Review of Linear Algebra (Cont)

                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                  bull Number of Elements in a Basis (Unique)

                                                  Review of Linear Algebra (Cont)

                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                  bull Number of Elements in a Basis (Unique)

                                                  bull (Use Basis Above)

                                                  bull eg dim of a line is 1

                                                  bull eg dim of a plane is 2

                                                  dd dim

                                                  Review of Linear Algebra (Cont)

                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                  bull Number of Elements in a Basis (Unique)

                                                  bull (Use Basis Above)

                                                  bull eg dim of a line is 1

                                                  bull eg dim of a plane is 2

                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                  (in Statistical Uses eg ANOVA)

                                                  dd dim

                                                  Review of Linear Algebra (Cont)

                                                  Norm of a Vector

                                                  bull in d 21

                                                  21

                                                  1

                                                  2 xxxx td

                                                  jj

                                                  Review of Linear Algebra (Cont)

                                                  Norm of a Vector

                                                  bull in

                                                  bull Idea length of the vector

                                                  d 2121

                                                  1

                                                  2 xxxx td

                                                  jj

                                                  Review of Linear Algebra (Cont)

                                                  Norm of a Vector

                                                  bull in

                                                  bull Idea length of the vector

                                                  bull Note strange properties for high

                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                  d 2121

                                                  1

                                                  2 xxxx td

                                                  jj

                                                  d

                                                  d

                                                  Review of Linear Algebra (Cont)

                                                  Norm of a Vector (cont)

                                                  bull Length Normalized Vector

                                                  (has Length 1 thus on Surf of Unit Sphere

                                                  amp is a Direction Vector)

                                                  x

                                                  x

                                                  Review of Linear Algebra (Cont)

                                                  Norm of a Vector (cont)

                                                  bull Length Normalized Vector

                                                  (has Length 1 thus on Surf of Unit Sphere

                                                  amp is a Direction Vector)

                                                  bull Define Distance as

                                                  x

                                                  x

                                                  yxyxyxyxd t

                                                  Review of Linear Algebra (Cont)

                                                  Inner (Dot Scalar) Product

                                                  bull for Vectors and

                                                  yxyxyx td

                                                  jjj

                                                  1

                                                  x y

                                                  Review of Linear Algebra (Cont)

                                                  Inner (Dot Scalar) Product

                                                  bull for Vectors and

                                                  bull Related to Norm via

                                                  yxyxyx td

                                                  jjj

                                                  1

                                                  x y

                                                  21 xxx

                                                  Review of Linear Algebra (Cont)

                                                  Inner (Dot Scalar) Product (cont)

                                                  bull measures ldquoangle between and rdquo as

                                                  yyxx

                                                  yx

                                                  yx

                                                  yxyxangle

                                                  tt

                                                  t

                                                  11 cos

                                                  cos

                                                  x y

                                                  Review of Linear Algebra (Cont)

                                                  Inner (Dot Scalar) Product (cont)

                                                  bull measures ldquoangle between and rdquo as

                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                  if and only if

                                                  yyxx

                                                  yx

                                                  yx

                                                  yxyxangle

                                                  tt

                                                  t

                                                  11 cos

                                                  cos

                                                  x y

                                                  yx 0 yx

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis

                                                  bull All Orthogonal to each other

                                                  ie for

                                                  bull All have Length 1

                                                  ie for

                                                  nvv 1

                                                  1 ii vv

                                                  0 ii vv ii

                                                  ni 1

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis (cont)

                                                  bull Spectral Representation

                                                  where

                                                  nvv 1

                                                  n

                                                  iii vax

                                                  1

                                                  ii vxa

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis (cont)

                                                  bull Spectral Representation

                                                  where

                                                  (Coefficient is Inner Product Cool Notation)

                                                  nvv 1

                                                  n

                                                  iii vax

                                                  1

                                                  ii vxa

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis (cont)

                                                  bull Spectral Representation

                                                  where

                                                  Check

                                                  nvv 1

                                                  n

                                                  iii vax

                                                  1

                                                  ii vxa

                                                  iii

                                                  n

                                                  iii

                                                  n

                                                  iiii avvavvavx

                                                  1

                                                  1

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis (cont)

                                                  bull Spectral Representation

                                                  where

                                                  Check

                                                  bull Matrix Notation where ie

                                                  For the Basis Matrix

                                                  nvv 1

                                                  n

                                                  iii vax

                                                  1

                                                  ii vxa

                                                  iii

                                                  n

                                                  iii

                                                  n

                                                  iiii avvavvavx

                                                  1

                                                  1

                                                  aBx Bxa tt xBa t

                                                  nvvB 1

                                                  Review of Linear Algebra (Cont)

                                                  Orthonormal Basis (cont)

                                                  bull Spectral Representation

                                                  where

                                                  Check

                                                  bull Matrix Notation where ie

                                                  is called transform of

                                                  (eg Fourier or Wavelet)

                                                  nvv 1

                                                  n

                                                  iii vax

                                                  1

                                                  ii vxa

                                                  iii

                                                  n

                                                  iii

                                                  n

                                                  iiii avvavvavx

                                                  1

                                                  1

                                                  aBx Bxa tt xBa t

                                                  xa

                                                  Review of Linear Algebra (Cont)

                                                  Parseval identity for

                                                  in subsp genrsquod by o n basis

                                                  x

                                                  nvv 1

                                                  2

                                                  1

                                                  22

                                                  1

                                                  2 aavxx

                                                  n

                                                  ii

                                                  n

                                                  ii

                                                  Review of Linear Algebra (Cont)

                                                  Parseval identity for

                                                  in subsp genrsquod by o n basis

                                                  bull Pythagorean theorem

                                                  bull ldquoDecomposition of Energyrdquo

                                                  bull ANOVA - sums of squares

                                                  x

                                                  nvv 1

                                                  2

                                                  1

                                                  22

                                                  1

                                                  2 aavxx

                                                  n

                                                  ii

                                                  n

                                                  ii

                                                  Review of Linear Algebra (Cont)

                                                  Parseval identity for

                                                  in subsp genrsquod by o n basis

                                                  bull Pythagorean theorem

                                                  bull ldquoDecomposition of Energyrdquo

                                                  bull ANOVA - sums of squares

                                                  bull Transform has same length as

                                                  ie ldquorotation in rdquo

                                                  x

                                                  nvv 1

                                                  2

                                                  1

                                                  22

                                                  1

                                                  2 aavxx

                                                  n

                                                  ii

                                                  n

                                                  ii

                                                  a xd

                                                  Projection of a Vector onto a Subspace

                                                  bull Idea Member of that is Closest to

                                                  (ie ldquoBest Approxrsquonrdquo)

                                                  Review of Linear Algebra (Cont)x

                                                  xV

                                                  V

                                                  Projection of a Vector onto a Subspace

                                                  bull Idea Member of that is Closest to

                                                  (ie ldquoBest Approxrsquonrdquo)

                                                  bull Find that Solves

                                                  (ldquoLeast Squaresrdquo)

                                                  Review of Linear Algebra (Cont)x

                                                  xV

                                                  V

                                                  VxPV vxVv

                                                  min

                                                  Projection of a Vector onto a Subspace

                                                  bull Idea Member of that is Closest to

                                                  (ie ldquoBest Approxrsquonrdquo)

                                                  bull Find that Solves

                                                  (ldquoLeast Squaresrdquo)

                                                  bull For Inner Product (Hilbert) Space

                                                  Exists and is Unique

                                                  Review of Linear Algebra (Cont)x

                                                  xV

                                                  V

                                                  VxPV vxVv

                                                  min

                                                  xPV

                                                  Projection of a Vector onto a Subspace (cont)

                                                  bull General Solution in for Basis Matrix

                                                  Review of Linear Algebra (Cont)

                                                  d VB

                                                  xBBBBxP tVV

                                                  tVVV

                                                  1

                                                  Projection of a Vector onto a Subspace (cont)

                                                  bull General Solution in for Basis Matrix

                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                  (thus projection is another linear operation)

                                                  Review of Linear Algebra (Cont)

                                                  d VB

                                                  xBBBBxP tVV

                                                  tVVV

                                                  1

                                                  tVV

                                                  tVVV BBBBP

                                                  1

                                                  Projection of a Vector onto a Subspace (cont)

                                                  bull General Solution in for Basis Matrix

                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                  (thus projection is another linear operation)

                                                  (note same operation underlies least squares)

                                                  Review of Linear Algebra (Cont)

                                                  d VB

                                                  xBBBBxP tVV

                                                  tVVV

                                                  1

                                                  tVV

                                                  tVVV BBBBP

                                                  1

                                                  Review of Linear Algebra (Cont)

                                                  Projection using Orthonormal Basis

                                                  bull Basis Matrix is Orthonormal

                                                  nnVtV IBB

                                                  10

                                                  01

                                                  1

                                                  111

                                                  1

                                                  1

                                                  nnn

                                                  n

                                                  ntn

                                                  t

                                                  vvvv

                                                  vvvv

                                                  vv

                                                  v

                                                  v

                                                  nvv 1

                                                  Review of Linear Algebra (Cont)

                                                  Projection using Orthonormal Basis

                                                  bull Basis Matrix is Orthonormal

                                                  bull So =

                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                  (Recall Right Multrsquon)

                                                  nnVtV IBB

                                                  10

                                                  01

                                                  1

                                                  111

                                                  1

                                                  1

                                                  nnn

                                                  n

                                                  ntn

                                                  t

                                                  vvvv

                                                  vvvv

                                                  vv

                                                  v

                                                  v

                                                  xBBxP tVVV

                                                  x V

                                                  nvv 1

                                                  Review of Linear Algebra (Cont)

                                                  Projection using Orthonormal Basis (cont)

                                                  bull For Orthogonal Complement

                                                  and

                                                  V

                                                  xPxPx VV 222xPxPx VV

                                                  Review of Linear Algebra (Cont)

                                                  Projection using Orthonormal Basis (cont)

                                                  bull For Orthogonal Complement

                                                  and

                                                  bull Parseval Inequality

                                                  V

                                                  xPxPx VV 222xPxPx VV

                                                  2

                                                  1

                                                  22

                                                  1

                                                  22 aavxxxP

                                                  n

                                                  ii

                                                  n

                                                  iiV

                                                  Review of Linear Algebra (Cont)

                                                  (Real) Unitary Matrices with

                                                  bull Orthonormal Basis Matrix

                                                  (So All of Above Applies)

                                                  ddU IUU t

                                                  Review of Linear Algebra (Cont)

                                                  (Real) Unitary Matrices with

                                                  bull Orthonormal Basis Matrix

                                                  (So All of Above Applies)

                                                  bull Note Transformrsquon is Distance Preserving

                                                  ddU IUU t

                                                  yxdyxyxyxUyUxUdn

                                                  i ii 2

                                                  1

                                                  Review of Linear Algebra (Cont)

                                                  (Real) Unitary Matrices with

                                                  bull Orthonormal Basis Matrix

                                                  (So All of Above Applies)

                                                  bull Note Transformrsquon is Distance Preserving

                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                  ddU IUU t

                                                  yxdyxyxyxUyUxUdn

                                                  i ii 2

                                                  1

                                                  U

                                                  Review of Linear Algebra (Cont)

                                                  Singular Value Decomposition (SVD)

                                                  For a Matrix

                                                  Find

                                                  ndX

                                                  Review of Linear Algebra (Cont)

                                                  Singular Value Decomposition (SVD)

                                                  For a Matrix

                                                  Find a Diagonal Matrix

                                                  with Entries

                                                  called Singular Values

                                                  ndX

                                                  ndS

                                                  )min(1 ndss

                                                  Review of Linear Algebra (Cont)

                                                  Singular Value Decomposition (SVD)

                                                  For a Matrix

                                                  Find a Diagonal Matrix

                                                  with Entries

                                                  called Singular Values

                                                  And Unitary (Rotation) Matrices

                                                  (recall )

                                                  ndX

                                                  ndS

                                                  )min(1 ndss

                                                  ddU nnV

                                                  IVVUU tt

                                                  Review of Linear Algebra (Cont)

                                                  Singular Value Decomposition (SVD)

                                                  For a Matrix

                                                  Find a Diagonal Matrix

                                                  with Entries

                                                  called Singular Values

                                                  And Unitary (Rotation) Matrices

                                                  (recall )

                                                  So That

                                                  ndX

                                                  ndS

                                                  )min(1 ndss

                                                  ddU nnV

                                                  IVVUU tt tUSVX

                                                  Review of Linear Algebra (Cont)

                                                  Intuition behind Singular Value Decomposition

                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                  vVSUvVSUvX tt

                                                  Review of Linear Algebra (Cont)

                                                  Intuition behind Singular Value Decomposition

                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                  bull First rotate

                                                  X

                                                  vVSUvVSUvX tt

                                                  Review of Linear Algebra (Cont)

                                                  Intuition behind Singular Value Decomposition

                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                  bull First rotate

                                                  bull Second rescale coordinate axes (by )

                                                  X

                                                  vVSUvVSUvX tt

                                                  is

                                                  Review of Linear Algebra (Cont)

                                                  Intuition behind Singular Value Decomposition

                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                  bull First rotate

                                                  bull Second rescale coordinate axes (by )

                                                  bull Third rotate again

                                                  X

                                                  vVSUvVSUvX tt

                                                  is

                                                  Review of Linear Algebra (Cont)

                                                  Intuition behind Singular Value Decomposition

                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                  bull First rotate

                                                  bull Second rescale coordinate axes (by )

                                                  bull Third rotate again

                                                  bull ie have diagonalized the transformation

                                                  X

                                                  vVSUvVSUvX tt

                                                  is

                                                  SVD Compact Representation

                                                  Useful Labeling

                                                  Singular Values in Increasing Order

                                                  Review of Linear Algebra (Cont)

                                                  )min(1 dnss

                                                  SVD Compact Representation

                                                  Useful Labeling

                                                  Singular Values in Increasing Order

                                                  Note singular values = 0 can be omitted

                                                  (Since do ldquo0-Stretchingrdquo)

                                                  Review of Linear Algebra (Cont)

                                                  )min(1 dnss

                                                  Review of Linear Algebra (Cont)

                                                  r

                                                  SVD Compact Representation

                                                  Useful Labeling

                                                  Singular Values in Increasing Order

                                                  Note singular values = 0 can be omitted

                                                  Let = of positive singular values

                                                  )min(1 dnss

                                                  Review of Linear Algebra (Cont)

                                                  r

                                                  SVD Compact Representation

                                                  Useful Labeling

                                                  Singular Values in Increasing Order

                                                  Note singular values = 0 can be omitted

                                                  Let = of positive singular values

                                                  Then

                                                  Where are truncations of

                                                  trnrrrd VSUX

                                                  VSU

                                                  )min(1 dnss

                                                  Review of Linear Algebra (Cont)

                                                  SVD Full Representation

                                                  =

                                                  Graphics Display Assumes

                                                  ndX ddU ndS nn

                                                  tV

                                                  nd

                                                  Review of Linear Algebra (Cont)

                                                  SVD Full Representation

                                                  =

                                                  Full Rank Basis Matrix

                                                  ndX ddU ndS nn

                                                  tV

                                                  Review of Linear Algebra (Cont)

                                                  SVD Full Representation

                                                  =

                                                  Full Rank Basis Matrix

                                                  All 0s in Bottom

                                                  ndX ddU ndS nn

                                                  tV

                                                  Review of Linear Algebra (Cont)

                                                  SVD Reduced Representation

                                                  =

                                                  These Columns Get 0ed Out

                                                  ndX ddU nnS nn

                                                  tV

                                                  nnd 0

                                                  Review of Linear Algebra (Cont)

                                                  SVD Reduced Representation

                                                  =ndX ndU nnS nn

                                                  tV

                                                  Review of Linear Algebra (Cont)

                                                  SVD Reduced Representation

                                                  =

                                                  Also Some of These May be 0

                                                  ndX ndU nnS nn

                                                  tV

                                                  Review of Linear Algebra (Cont)

                                                  SVD Compact Representation

                                                  = ndX rdU

                                                  rrS nrtV

                                                  0

                                                  Review of Linear Algebra (Cont)

                                                  SVD Compact Representation

                                                  =

                                                  These Get 0ed Out

                                                  ndX rdU

                                                  rrS nrtV

                                                  0

                                                  Review of Linear Algebra (Cont)

                                                  SVD Compact Representation

                                                  = ndX rdU

                                                  rrS nrtV

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition

                                                  For a (Symmetric) Square Matrix

                                                  Find

                                                  ddX

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition

                                                  For a (Symmetric) Square Matrix

                                                  Find a Diagonal Matrix

                                                  Called Eigenvalues

                                                  Convenient Ordering

                                                  ddX

                                                  d

                                                  D

                                                  0

                                                  01

                                                  n 1

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition

                                                  For a (Symmetric) Square Matrix

                                                  Find a Diagonal Matrix

                                                  And an Orthonormal Matrix

                                                  (ie )

                                                  ddX

                                                  d

                                                  D

                                                  0

                                                  01

                                                  ddB

                                                  ddtt IBBBB

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition

                                                  For a (Symmetric) Square Matrix

                                                  Find a Diagonal Matrix

                                                  And an Orthonormal Matrix

                                                  (ie )

                                                  So that ie

                                                  ddX

                                                  d

                                                  D

                                                  0

                                                  01

                                                  ddB

                                                  ddtt IBBBB

                                                  DBBX tBDBX

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                  (looks similar)

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                  Complex (uses )

                                                  VU

                                                  1i

                                                  Review of Linear Algebra (Cont)

                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                  VU

                                                  X

                                                  BVU

                                                  1i

                                                  • Participant Presentations
                                                  • PCA to find clusters
                                                  • Statistical Smoothing
                                                  • Kernel Density Estimation
                                                  • Scatterplot Smoothing
                                                  • Statistical Smoothing (2)
                                                  • SiZer Background
                                                  • SiZer Background (2)
                                                  • SiZer Background (3)
                                                  • SiZer Background (4)
                                                  • SiZer Background (5)
                                                  • SiZer Overview
                                                  • PCA to find clusters (2)
                                                  • PCA to find clusters (3)
                                                  • PCA to find clusters (4)
                                                  • PCA to find clusters (5)
                                                  • PCA to find clusters (6)
                                                  • PCA to find clusters (7)
                                                  • Recall Yeast Cell Cycle Data
                                                  • Yeast Cell Cycle Data FDA View
                                                  • Yeast Cell Cycle Data FDA View (2)
                                                  • Yeast Cell Cycle Data FDA View (3)
                                                  • Yeast Cell Cycles Freq 2 Proj
                                                  • Frequency 2 Analysis
                                                  • Frequency 2 Analysis (2)
                                                  • Frequency 2 Analysis (3)
                                                  • Yeast Cell Cycle
                                                  • SiZer Study of Distrsquon of Angles
                                                  • Reclassification of Major Genes
                                                  • Compare to Previous Classifrsquon
                                                  • New Subpopulation View
                                                  • New Subpopulation View (2)
                                                  • Detailed Look at PCA
                                                  • PCA Rediscovery ndash Renaming
                                                  • PCA Rediscovery ndash Renaming (2)
                                                  • PCA Rediscovery ndash Renaming (3)
                                                  • PCA Rediscovery ndash Renaming (4)
                                                  • PCA Rediscovery ndash Renaming (5)
                                                  • An Interesting Historical Note
                                                  • An Interesting Historical Note (2)
                                                  • Detailed Look at PCA (2)
                                                  • Detailed Look at PCA (3)
                                                  • Review of Linear Algebra
                                                  • Review of Linear Algebra (2)
                                                  • Review of Linear Algebra (3)
                                                  • Review of Linear Algebra (Cont)
                                                  • Review of Linear Algebra (Cont) (2)
                                                  • Review of Linear Algebra (Cont) (3)
                                                  • Review of Linear Algebra (Cont) (4)
                                                  • Review of Linear Algebra (Cont) (5)
                                                  • Review of Linear Algebra (Cont) (6)
                                                  • Review of Linear Algebra (Cont) (7)
                                                  • Review of Linear Algebra (Cont) (8)
                                                  • Review of Linear Algebra (Cont) (9)
                                                  • Review of Linear Algebra (Cont) (10)
                                                  • Review of Linear Algebra (Cont) (11)
                                                  • Review of Linear Algebra (Cont) (12)
                                                  • Review of Linear Algebra (Cont) (13)
                                                  • Review of Linear Algebra (Cont) (14)
                                                  • Review of Linear Algebra (Cont) (15)
                                                  • Review of Linear Algebra (Cont) (16)
                                                  • Review of Linear Algebra (Cont) (17)
                                                  • Review of Linear Algebra (Cont) (18)
                                                  • Review of Linear Algebra (Cont) (19)
                                                  • Review of Linear Algebra (Cont) (20)
                                                  • Review of Linear Algebra (Cont) (21)
                                                  • Review of Linear Algebra (Cont) (22)
                                                  • Review of Linear Algebra (Cont) (23)
                                                  • Review of Linear Algebra (Cont) (24)
                                                  • Review of Linear Algebra (Cont) (25)
                                                  • Review of Linear Algebra (Cont) (26)
                                                  • Review of Linear Algebra (Cont) (27)
                                                  • Review of Linear Algebra (Cont) (28)
                                                  • Review of Linear Algebra (Cont) (29)
                                                  • Review of Linear Algebra (Cont) (30)
                                                  • Review of Linear Algebra (Cont) (31)
                                                  • Review of Linear Algebra (Cont) (32)
                                                  • Review of Linear Algebra (Cont) (33)
                                                  • Review of Linear Algebra (Cont) (34)
                                                  • Review of Linear Algebra (Cont) (35)
                                                  • Review of Linear Algebra (Cont) (36)
                                                  • Review of Linear Algebra (Cont) (37)
                                                  • Review of Linear Algebra (Cont) (38)
                                                  • Review of Linear Algebra (Cont) (39)
                                                  • Review of Linear Algebra (Cont) (40)
                                                  • Review of Linear Algebra (Cont) (41)
                                                  • Review of Linear Algebra (Cont) (42)
                                                  • Review of Linear Algebra (Cont) (43)
                                                  • Review of Linear Algebra (Cont) (44)
                                                  • Review of Linear Algebra (Cont) (45)
                                                  • Review of Linear Algebra (Cont) (46)
                                                  • Review of Linear Algebra (Cont) (47)
                                                  • Review of Linear Algebra (Cont) (48)
                                                  • Review of Linear Algebra (Cont) (49)
                                                  • Review of Linear Algebra (Cont) (50)
                                                  • Review of Linear Algebra (Cont) (51)
                                                  • Review of Linear Algebra (Cont) (52)
                                                  • Review of Linear Algebra (Cont) (53)
                                                  • Review of Linear Algebra (Cont) (54)
                                                  • Review of Linear Algebra (Cont) (55)
                                                  • Review of Linear Algebra (Cont) (56)
                                                  • Review of Linear Algebra (Cont) (57)
                                                  • Review of Linear Algebra (Cont) (58)
                                                  • Review of Linear Algebra (Cont) (59)
                                                  • Review of Linear Algebra (Cont) (60)
                                                  • Review of Linear Algebra (Cont) (61)
                                                  • Review of Linear Algebra (Cont) (62)
                                                  • Review of Linear Algebra (Cont) (63)
                                                  • Review of Linear Algebra (Cont) (64)
                                                  • Review of Linear Algebra (Cont) (65)
                                                  • Review of Linear Algebra (Cont) (66)
                                                  • Review of Linear Algebra (Cont) (67)
                                                  • Review of Linear Algebra (Cont) (68)
                                                  • Review of Linear Algebra (Cont) (69)
                                                  • Review of Linear Algebra (Cont) (70)
                                                  • Review of Linear Algebra (Cont) (71)
                                                  • Review of Linear Algebra (Cont) (72)
                                                  • Review of Linear Algebra (Cont) (73)
                                                  • Review of Linear Algebra (Cont) (74)
                                                  • Review of Linear Algebra (Cont) (75)
                                                  • Review of Linear Algebra (Cont) (76)
                                                  • Review of Linear Algebra (Cont) (77)
                                                  • Review of Linear Algebra (Cont) (78)
                                                  • Review of Linear Algebra (Cont) (79)
                                                  • Review of Linear Algebra (Cont) (80)
                                                  • Review of Linear Algebra (Cont) (81)

                                                    Frequency 2 Analysisbull Project data onto 2-dim space of sin and

                                                    cos (freq 2)bull Useful view scatterplotbull Angle (in polar coordinates) shows phasebull Colors Spellmanrsquos cell cycle phase

                                                    classificationbull Black was labeled ldquonot periodicrdquobull Within class phases approxrsquoly same but

                                                    notable differencesbull Now try to improve ldquophase classificationrdquo

                                                    Yeast Cell CycleRevisit ldquophase classificationrdquo

                                                    approachbull Use outer 200 genes

                                                    (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                                    (finds significant bumps etc in histogram)

                                                    bull Carefully redrew boundariesbull Check by studying kde angles

                                                    SiZer Study of Distrsquon of Angles

                                                    Reclassification of Major Genes

                                                    Compare to Previous Classifrsquon

                                                    New Subpopulation View

                                                    New Subpopulation View

                                                    NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                    Detailed Look at PCA

                                                    Now Study ldquoFolklorerdquo More Carefully

                                                    bull BackGround

                                                    bull History

                                                    bull Underpinnings

                                                    (Mathematical amp Computational)

                                                    Good Overall Reference Jolliffe (2002)

                                                    PCA Rediscovery ndash Renaming

                                                    Statistics Principal Component Analysis (PCA)

                                                    PCA Rediscovery ndash Renaming

                                                    Statistics Principal Component Analysis (PCA)

                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                    PCA Rediscovery ndash Renaming

                                                    Statistics Principal Component Analysis (PCA)

                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                    PCA Rediscovery ndash Renaming

                                                    Statistics Principal Component Analysis (PCA)

                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                                    PCA Rediscovery ndash Renaming

                                                    Statistics Principal Component Analysis (PCA)

                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                                    Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                    An Interesting Historical Note

                                                    The 1st () application of PCA to Functional

                                                    Data Analysis

                                                    An Interesting Historical Note

                                                    The 1st () application of PCA to Functional

                                                    Data Analysis

                                                    Rao (1958)

                                                    1st Paper with ldquoCurves as Data Objectsrdquo

                                                    viewpoint

                                                    Detailed Look at PCA

                                                    Three Important (amp Interesting) Viewpoints

                                                    1 Mathematics

                                                    2 Numerics

                                                    3 Statistics

                                                    Goal Study Interrelationships

                                                    Detailed Look at PCA

                                                    Three Important (amp Interesting) Viewpoints

                                                    1 Mathematics

                                                    2 Numerics

                                                    3 Statistics

                                                    1st Review Linear Alg and Multivar Prob

                                                    Review of Linear Algebra

                                                    Vector Space

                                                    bull set of ldquovectorsrdquo

                                                    bull and ldquoscalarsrdquo (coefficients)

                                                    xa

                                                    Review of Linear Algebra

                                                    Vector Space

                                                    bull set of ldquovectorsrdquo

                                                    bull and ldquoscalarsrdquo (coefficients)

                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                    ( in space)

                                                    xa

                                                    i

                                                    ii xa

                                                    Review of Linear Algebra

                                                    Vector Space

                                                    bull set of ldquovectorsrdquo

                                                    bull and ldquoscalarsrdquo (coefficients)

                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                    ( in space)

                                                    eg

                                                    ldquo dim Euclidrsquon spacerdquo

                                                    xa

                                                    i

                                                    ii xa

                                                    d

                                                    d

                                                    d xx

                                                    x

                                                    x

                                                    x 1

                                                    1

                                                    d

                                                    Review of Linear Algebra (Cont)

                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                    Review of Linear Algebra (Cont)

                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                    Review of Linear Algebra (Cont)

                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                    Note Planes not Through the Origin

                                                    are not Subspaces

                                                    (Do not Contain )00 x

                                                    Review of Linear Algebra (Cont)

                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                    (all Linear Combos of them =

                                                    = Containing Hyperplane

                                                    through Origin)

                                                    Review of Linear Algebra (Cont)

                                                    Basis of Subspace Set of Vectors that

                                                    bull Span ie Everything is a Lin Com of them

                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                    Review of Linear Algebra (Cont)

                                                    Basis of Subspace Set of Vectors that

                                                    bull Span ie Everything is a Lin Com of them

                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                    bull eg ldquoUnit Vector Basisrdquo

                                                    bull Since

                                                    d

                                                    1

                                                    0

                                                    0

                                                    0

                                                    1

                                                    0

                                                    0

                                                    0

                                                    1

                                                    1

                                                    0

                                                    0

                                                    0

                                                    1

                                                    0

                                                    0

                                                    0

                                                    1

                                                    212

                                                    1

                                                    d

                                                    d

                                                    xxx

                                                    x

                                                    x

                                                    x

                                                    Review of Linear Algebra (Cont)

                                                    Basis Matrix of subspace of

                                                    Given a basis

                                                    create matrix of columns

                                                    dnvv 1

                                                    nddnd

                                                    n

                                                    n

                                                    vv

                                                    vv

                                                    vvB

                                                    1

                                                    111

                                                    1

                                                    Review of Linear Algebra (Cont)

                                                    Then linear combo is a matrix multiplicatrsquon

                                                    where

                                                    n

                                                    iii aBva

                                                    1

                                                    na

                                                    a

                                                    a 1

                                                    Review of Linear Algebra (Cont)

                                                    Then linear combo is a matrix multiplicatrsquon

                                                    where

                                                    Note Right Multiplication Gives

                                                    Linear Combination of Column Vectors

                                                    n

                                                    iii aBva

                                                    1

                                                    na

                                                    a

                                                    a 1

                                                    Review of Linear Algebra (Cont)

                                                    Then linear combo is a matrix multiplicatrsquon

                                                    where

                                                    Check sizes

                                                    n

                                                    iii aBva

                                                    1

                                                    na

                                                    a

                                                    a 1

                                                    )1()(1 nndd

                                                    Review of Linear Algebra (Cont)

                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                    For matrices

                                                    Define the Matrix Product

                                                    mkk

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    nmm

                                                    n

                                                    bb

                                                    bb

                                                    B

                                                    1

                                                    111

                                                    m

                                                    iniik

                                                    m

                                                    iiik

                                                    m

                                                    inii

                                                    m

                                                    iii

                                                    baba

                                                    baba

                                                    AB

                                                    1

                                                    11

                                                    11

                                                    111

                                                    Review of Linear Algebra (Cont)

                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                    For matrices

                                                    Define the Matrix Product

                                                    (Inner Products of Rows With Columns )

                                                    mkk

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    nmm

                                                    n

                                                    bb

                                                    bb

                                                    B

                                                    1

                                                    111

                                                    m

                                                    iniik

                                                    m

                                                    iiik

                                                    m

                                                    inii

                                                    m

                                                    iii

                                                    baba

                                                    baba

                                                    AB

                                                    1

                                                    11

                                                    11

                                                    111

                                                    A B

                                                    Review of Linear Algebra (Cont)

                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                    For matrices

                                                    Define the Matrix Product

                                                    (Inner Products of Rows With Columns )

                                                    (Composition of Linear Transformations)

                                                    mkk

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    nmm

                                                    n

                                                    bb

                                                    bb

                                                    B

                                                    1

                                                    111

                                                    m

                                                    iniik

                                                    m

                                                    iiik

                                                    m

                                                    inii

                                                    m

                                                    iii

                                                    baba

                                                    baba

                                                    AB

                                                    1

                                                    11

                                                    11

                                                    111

                                                    A B

                                                    Review of Linear Algebra (Cont)

                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                    For matrices

                                                    Define the Matrix Product

                                                    (Inner Products of Rows With Columns )

                                                    (Composition of Linear Transformations)

                                                    Often Useful to Check Sizes

                                                    mkk

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    nmm

                                                    n

                                                    bb

                                                    bb

                                                    B

                                                    1

                                                    111

                                                    m

                                                    iniik

                                                    m

                                                    iiik

                                                    m

                                                    inii

                                                    m

                                                    iii

                                                    baba

                                                    baba

                                                    AB

                                                    1

                                                    11

                                                    11

                                                    111

                                                    nmmknk

                                                    A B

                                                    Review of Linear Algebra (Cont)

                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                    For matrices

                                                    Define the Matrix Product

                                                    (Inner Products of Rows With Columns )

                                                    (Composition of Linear Transformations)

                                                    Often Useful to Check Sizes

                                                    mkk

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    nmm

                                                    n

                                                    bb

                                                    bb

                                                    B

                                                    1

                                                    111

                                                    m

                                                    iniik

                                                    m

                                                    iiik

                                                    m

                                                    inii

                                                    m

                                                    iii

                                                    baba

                                                    baba

                                                    AB

                                                    1

                                                    11

                                                    11

                                                    111

                                                    nmmknk

                                                    A B

                                                    Review of Linear Algebra (Cont)

                                                    Matrix Trace

                                                    bull For a Square Matrix

                                                    bull Define

                                                    mmm

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    m

                                                    iiiaAtr

                                                    1)(

                                                    Review of Linear Algebra (Cont)

                                                    Matrix Trace

                                                    bull For a Square Matrix

                                                    bull Define

                                                    bull Trace Commutes with Matrix Multiplication

                                                    mmm

                                                    m

                                                    aa

                                                    aa

                                                    A

                                                    1

                                                    111

                                                    m

                                                    iiiaAtr

                                                    1)(

                                                    BAtrABtr

                                                    Review of Linear Algebra (Cont)

                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                    bull Number of Elements in a Basis (Unique)

                                                    Review of Linear Algebra (Cont)

                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                    bull Number of Elements in a Basis (Unique)

                                                    bull (Use Basis Above)

                                                    bull eg dim of a line is 1

                                                    bull eg dim of a plane is 2

                                                    dd dim

                                                    Review of Linear Algebra (Cont)

                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                    bull Number of Elements in a Basis (Unique)

                                                    bull (Use Basis Above)

                                                    bull eg dim of a line is 1

                                                    bull eg dim of a plane is 2

                                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                                    (in Statistical Uses eg ANOVA)

                                                    dd dim

                                                    Review of Linear Algebra (Cont)

                                                    Norm of a Vector

                                                    bull in d 21

                                                    21

                                                    1

                                                    2 xxxx td

                                                    jj

                                                    Review of Linear Algebra (Cont)

                                                    Norm of a Vector

                                                    bull in

                                                    bull Idea length of the vector

                                                    d 2121

                                                    1

                                                    2 xxxx td

                                                    jj

                                                    Review of Linear Algebra (Cont)

                                                    Norm of a Vector

                                                    bull in

                                                    bull Idea length of the vector

                                                    bull Note strange properties for high

                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                    d 2121

                                                    1

                                                    2 xxxx td

                                                    jj

                                                    d

                                                    d

                                                    Review of Linear Algebra (Cont)

                                                    Norm of a Vector (cont)

                                                    bull Length Normalized Vector

                                                    (has Length 1 thus on Surf of Unit Sphere

                                                    amp is a Direction Vector)

                                                    x

                                                    x

                                                    Review of Linear Algebra (Cont)

                                                    Norm of a Vector (cont)

                                                    bull Length Normalized Vector

                                                    (has Length 1 thus on Surf of Unit Sphere

                                                    amp is a Direction Vector)

                                                    bull Define Distance as

                                                    x

                                                    x

                                                    yxyxyxyxd t

                                                    Review of Linear Algebra (Cont)

                                                    Inner (Dot Scalar) Product

                                                    bull for Vectors and

                                                    yxyxyx td

                                                    jjj

                                                    1

                                                    x y

                                                    Review of Linear Algebra (Cont)

                                                    Inner (Dot Scalar) Product

                                                    bull for Vectors and

                                                    bull Related to Norm via

                                                    yxyxyx td

                                                    jjj

                                                    1

                                                    x y

                                                    21 xxx

                                                    Review of Linear Algebra (Cont)

                                                    Inner (Dot Scalar) Product (cont)

                                                    bull measures ldquoangle between and rdquo as

                                                    yyxx

                                                    yx

                                                    yx

                                                    yxyxangle

                                                    tt

                                                    t

                                                    11 cos

                                                    cos

                                                    x y

                                                    Review of Linear Algebra (Cont)

                                                    Inner (Dot Scalar) Product (cont)

                                                    bull measures ldquoangle between and rdquo as

                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                    if and only if

                                                    yyxx

                                                    yx

                                                    yx

                                                    yxyxangle

                                                    tt

                                                    t

                                                    11 cos

                                                    cos

                                                    x y

                                                    yx 0 yx

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis

                                                    bull All Orthogonal to each other

                                                    ie for

                                                    bull All have Length 1

                                                    ie for

                                                    nvv 1

                                                    1 ii vv

                                                    0 ii vv ii

                                                    ni 1

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis (cont)

                                                    bull Spectral Representation

                                                    where

                                                    nvv 1

                                                    n

                                                    iii vax

                                                    1

                                                    ii vxa

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis (cont)

                                                    bull Spectral Representation

                                                    where

                                                    (Coefficient is Inner Product Cool Notation)

                                                    nvv 1

                                                    n

                                                    iii vax

                                                    1

                                                    ii vxa

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis (cont)

                                                    bull Spectral Representation

                                                    where

                                                    Check

                                                    nvv 1

                                                    n

                                                    iii vax

                                                    1

                                                    ii vxa

                                                    iii

                                                    n

                                                    iii

                                                    n

                                                    iiii avvavvavx

                                                    1

                                                    1

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis (cont)

                                                    bull Spectral Representation

                                                    where

                                                    Check

                                                    bull Matrix Notation where ie

                                                    For the Basis Matrix

                                                    nvv 1

                                                    n

                                                    iii vax

                                                    1

                                                    ii vxa

                                                    iii

                                                    n

                                                    iii

                                                    n

                                                    iiii avvavvavx

                                                    1

                                                    1

                                                    aBx Bxa tt xBa t

                                                    nvvB 1

                                                    Review of Linear Algebra (Cont)

                                                    Orthonormal Basis (cont)

                                                    bull Spectral Representation

                                                    where

                                                    Check

                                                    bull Matrix Notation where ie

                                                    is called transform of

                                                    (eg Fourier or Wavelet)

                                                    nvv 1

                                                    n

                                                    iii vax

                                                    1

                                                    ii vxa

                                                    iii

                                                    n

                                                    iii

                                                    n

                                                    iiii avvavvavx

                                                    1

                                                    1

                                                    aBx Bxa tt xBa t

                                                    xa

                                                    Review of Linear Algebra (Cont)

                                                    Parseval identity for

                                                    in subsp genrsquod by o n basis

                                                    x

                                                    nvv 1

                                                    2

                                                    1

                                                    22

                                                    1

                                                    2 aavxx

                                                    n

                                                    ii

                                                    n

                                                    ii

                                                    Review of Linear Algebra (Cont)

                                                    Parseval identity for

                                                    in subsp genrsquod by o n basis

                                                    bull Pythagorean theorem

                                                    bull ldquoDecomposition of Energyrdquo

                                                    bull ANOVA - sums of squares

                                                    x

                                                    nvv 1

                                                    2

                                                    1

                                                    22

                                                    1

                                                    2 aavxx

                                                    n

                                                    ii

                                                    n

                                                    ii

                                                    Review of Linear Algebra (Cont)

                                                    Parseval identity for

                                                    in subsp genrsquod by o n basis

                                                    bull Pythagorean theorem

                                                    bull ldquoDecomposition of Energyrdquo

                                                    bull ANOVA - sums of squares

                                                    bull Transform has same length as

                                                    ie ldquorotation in rdquo

                                                    x

                                                    nvv 1

                                                    2

                                                    1

                                                    22

                                                    1

                                                    2 aavxx

                                                    n

                                                    ii

                                                    n

                                                    ii

                                                    a xd

                                                    Projection of a Vector onto a Subspace

                                                    bull Idea Member of that is Closest to

                                                    (ie ldquoBest Approxrsquonrdquo)

                                                    Review of Linear Algebra (Cont)x

                                                    xV

                                                    V

                                                    Projection of a Vector onto a Subspace

                                                    bull Idea Member of that is Closest to

                                                    (ie ldquoBest Approxrsquonrdquo)

                                                    bull Find that Solves

                                                    (ldquoLeast Squaresrdquo)

                                                    Review of Linear Algebra (Cont)x

                                                    xV

                                                    V

                                                    VxPV vxVv

                                                    min

                                                    Projection of a Vector onto a Subspace

                                                    bull Idea Member of that is Closest to

                                                    (ie ldquoBest Approxrsquonrdquo)

                                                    bull Find that Solves

                                                    (ldquoLeast Squaresrdquo)

                                                    bull For Inner Product (Hilbert) Space

                                                    Exists and is Unique

                                                    Review of Linear Algebra (Cont)x

                                                    xV

                                                    V

                                                    VxPV vxVv

                                                    min

                                                    xPV

                                                    Projection of a Vector onto a Subspace (cont)

                                                    bull General Solution in for Basis Matrix

                                                    Review of Linear Algebra (Cont)

                                                    d VB

                                                    xBBBBxP tVV

                                                    tVVV

                                                    1

                                                    Projection of a Vector onto a Subspace (cont)

                                                    bull General Solution in for Basis Matrix

                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                    (thus projection is another linear operation)

                                                    Review of Linear Algebra (Cont)

                                                    d VB

                                                    xBBBBxP tVV

                                                    tVVV

                                                    1

                                                    tVV

                                                    tVVV BBBBP

                                                    1

                                                    Projection of a Vector onto a Subspace (cont)

                                                    bull General Solution in for Basis Matrix

                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                    (thus projection is another linear operation)

                                                    (note same operation underlies least squares)

                                                    Review of Linear Algebra (Cont)

                                                    d VB

                                                    xBBBBxP tVV

                                                    tVVV

                                                    1

                                                    tVV

                                                    tVVV BBBBP

                                                    1

                                                    Review of Linear Algebra (Cont)

                                                    Projection using Orthonormal Basis

                                                    bull Basis Matrix is Orthonormal

                                                    nnVtV IBB

                                                    10

                                                    01

                                                    1

                                                    111

                                                    1

                                                    1

                                                    nnn

                                                    n

                                                    ntn

                                                    t

                                                    vvvv

                                                    vvvv

                                                    vv

                                                    v

                                                    v

                                                    nvv 1

                                                    Review of Linear Algebra (Cont)

                                                    Projection using Orthonormal Basis

                                                    bull Basis Matrix is Orthonormal

                                                    bull So =

                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                    (Recall Right Multrsquon)

                                                    nnVtV IBB

                                                    10

                                                    01

                                                    1

                                                    111

                                                    1

                                                    1

                                                    nnn

                                                    n

                                                    ntn

                                                    t

                                                    vvvv

                                                    vvvv

                                                    vv

                                                    v

                                                    v

                                                    xBBxP tVVV

                                                    x V

                                                    nvv 1

                                                    Review of Linear Algebra (Cont)

                                                    Projection using Orthonormal Basis (cont)

                                                    bull For Orthogonal Complement

                                                    and

                                                    V

                                                    xPxPx VV 222xPxPx VV

                                                    Review of Linear Algebra (Cont)

                                                    Projection using Orthonormal Basis (cont)

                                                    bull For Orthogonal Complement

                                                    and

                                                    bull Parseval Inequality

                                                    V

                                                    xPxPx VV 222xPxPx VV

                                                    2

                                                    1

                                                    22

                                                    1

                                                    22 aavxxxP

                                                    n

                                                    ii

                                                    n

                                                    iiV

                                                    Review of Linear Algebra (Cont)

                                                    (Real) Unitary Matrices with

                                                    bull Orthonormal Basis Matrix

                                                    (So All of Above Applies)

                                                    ddU IUU t

                                                    Review of Linear Algebra (Cont)

                                                    (Real) Unitary Matrices with

                                                    bull Orthonormal Basis Matrix

                                                    (So All of Above Applies)

                                                    bull Note Transformrsquon is Distance Preserving

                                                    ddU IUU t

                                                    yxdyxyxyxUyUxUdn

                                                    i ii 2

                                                    1

                                                    Review of Linear Algebra (Cont)

                                                    (Real) Unitary Matrices with

                                                    bull Orthonormal Basis Matrix

                                                    (So All of Above Applies)

                                                    bull Note Transformrsquon is Distance Preserving

                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                    ddU IUU t

                                                    yxdyxyxyxUyUxUdn

                                                    i ii 2

                                                    1

                                                    U

                                                    Review of Linear Algebra (Cont)

                                                    Singular Value Decomposition (SVD)

                                                    For a Matrix

                                                    Find

                                                    ndX

                                                    Review of Linear Algebra (Cont)

                                                    Singular Value Decomposition (SVD)

                                                    For a Matrix

                                                    Find a Diagonal Matrix

                                                    with Entries

                                                    called Singular Values

                                                    ndX

                                                    ndS

                                                    )min(1 ndss

                                                    Review of Linear Algebra (Cont)

                                                    Singular Value Decomposition (SVD)

                                                    For a Matrix

                                                    Find a Diagonal Matrix

                                                    with Entries

                                                    called Singular Values

                                                    And Unitary (Rotation) Matrices

                                                    (recall )

                                                    ndX

                                                    ndS

                                                    )min(1 ndss

                                                    ddU nnV

                                                    IVVUU tt

                                                    Review of Linear Algebra (Cont)

                                                    Singular Value Decomposition (SVD)

                                                    For a Matrix

                                                    Find a Diagonal Matrix

                                                    with Entries

                                                    called Singular Values

                                                    And Unitary (Rotation) Matrices

                                                    (recall )

                                                    So That

                                                    ndX

                                                    ndS

                                                    )min(1 ndss

                                                    ddU nnV

                                                    IVVUU tt tUSVX

                                                    Review of Linear Algebra (Cont)

                                                    Intuition behind Singular Value Decomposition

                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                    vVSUvVSUvX tt

                                                    Review of Linear Algebra (Cont)

                                                    Intuition behind Singular Value Decomposition

                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                    bull First rotate

                                                    X

                                                    vVSUvVSUvX tt

                                                    Review of Linear Algebra (Cont)

                                                    Intuition behind Singular Value Decomposition

                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                    bull First rotate

                                                    bull Second rescale coordinate axes (by )

                                                    X

                                                    vVSUvVSUvX tt

                                                    is

                                                    Review of Linear Algebra (Cont)

                                                    Intuition behind Singular Value Decomposition

                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                    bull First rotate

                                                    bull Second rescale coordinate axes (by )

                                                    bull Third rotate again

                                                    X

                                                    vVSUvVSUvX tt

                                                    is

                                                    Review of Linear Algebra (Cont)

                                                    Intuition behind Singular Value Decomposition

                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                    bull First rotate

                                                    bull Second rescale coordinate axes (by )

                                                    bull Third rotate again

                                                    bull ie have diagonalized the transformation

                                                    X

                                                    vVSUvVSUvX tt

                                                    is

                                                    SVD Compact Representation

                                                    Useful Labeling

                                                    Singular Values in Increasing Order

                                                    Review of Linear Algebra (Cont)

                                                    )min(1 dnss

                                                    SVD Compact Representation

                                                    Useful Labeling

                                                    Singular Values in Increasing Order

                                                    Note singular values = 0 can be omitted

                                                    (Since do ldquo0-Stretchingrdquo)

                                                    Review of Linear Algebra (Cont)

                                                    )min(1 dnss

                                                    Review of Linear Algebra (Cont)

                                                    r

                                                    SVD Compact Representation

                                                    Useful Labeling

                                                    Singular Values in Increasing Order

                                                    Note singular values = 0 can be omitted

                                                    Let = of positive singular values

                                                    )min(1 dnss

                                                    Review of Linear Algebra (Cont)

                                                    r

                                                    SVD Compact Representation

                                                    Useful Labeling

                                                    Singular Values in Increasing Order

                                                    Note singular values = 0 can be omitted

                                                    Let = of positive singular values

                                                    Then

                                                    Where are truncations of

                                                    trnrrrd VSUX

                                                    VSU

                                                    )min(1 dnss

                                                    Review of Linear Algebra (Cont)

                                                    SVD Full Representation

                                                    =

                                                    Graphics Display Assumes

                                                    ndX ddU ndS nn

                                                    tV

                                                    nd

                                                    Review of Linear Algebra (Cont)

                                                    SVD Full Representation

                                                    =

                                                    Full Rank Basis Matrix

                                                    ndX ddU ndS nn

                                                    tV

                                                    Review of Linear Algebra (Cont)

                                                    SVD Full Representation

                                                    =

                                                    Full Rank Basis Matrix

                                                    All 0s in Bottom

                                                    ndX ddU ndS nn

                                                    tV

                                                    Review of Linear Algebra (Cont)

                                                    SVD Reduced Representation

                                                    =

                                                    These Columns Get 0ed Out

                                                    ndX ddU nnS nn

                                                    tV

                                                    nnd 0

                                                    Review of Linear Algebra (Cont)

                                                    SVD Reduced Representation

                                                    =ndX ndU nnS nn

                                                    tV

                                                    Review of Linear Algebra (Cont)

                                                    SVD Reduced Representation

                                                    =

                                                    Also Some of These May be 0

                                                    ndX ndU nnS nn

                                                    tV

                                                    Review of Linear Algebra (Cont)

                                                    SVD Compact Representation

                                                    = ndX rdU

                                                    rrS nrtV

                                                    0

                                                    Review of Linear Algebra (Cont)

                                                    SVD Compact Representation

                                                    =

                                                    These Get 0ed Out

                                                    ndX rdU

                                                    rrS nrtV

                                                    0

                                                    Review of Linear Algebra (Cont)

                                                    SVD Compact Representation

                                                    = ndX rdU

                                                    rrS nrtV

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition

                                                    For a (Symmetric) Square Matrix

                                                    Find

                                                    ddX

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition

                                                    For a (Symmetric) Square Matrix

                                                    Find a Diagonal Matrix

                                                    Called Eigenvalues

                                                    Convenient Ordering

                                                    ddX

                                                    d

                                                    D

                                                    0

                                                    01

                                                    n 1

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition

                                                    For a (Symmetric) Square Matrix

                                                    Find a Diagonal Matrix

                                                    And an Orthonormal Matrix

                                                    (ie )

                                                    ddX

                                                    d

                                                    D

                                                    0

                                                    01

                                                    ddB

                                                    ddtt IBBBB

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition

                                                    For a (Symmetric) Square Matrix

                                                    Find a Diagonal Matrix

                                                    And an Orthonormal Matrix

                                                    (ie )

                                                    So that ie

                                                    ddX

                                                    d

                                                    D

                                                    0

                                                    01

                                                    ddB

                                                    ddtt IBBBB

                                                    DBBX tBDBX

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                    (looks similar)

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                    Complex (uses )

                                                    VU

                                                    1i

                                                    Review of Linear Algebra (Cont)

                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                    VU

                                                    X

                                                    BVU

                                                    1i

                                                    • Participant Presentations
                                                    • PCA to find clusters
                                                    • Statistical Smoothing
                                                    • Kernel Density Estimation
                                                    • Scatterplot Smoothing
                                                    • Statistical Smoothing (2)
                                                    • SiZer Background
                                                    • SiZer Background (2)
                                                    • SiZer Background (3)
                                                    • SiZer Background (4)
                                                    • SiZer Background (5)
                                                    • SiZer Overview
                                                    • PCA to find clusters (2)
                                                    • PCA to find clusters (3)
                                                    • PCA to find clusters (4)
                                                    • PCA to find clusters (5)
                                                    • PCA to find clusters (6)
                                                    • PCA to find clusters (7)
                                                    • Recall Yeast Cell Cycle Data
                                                    • Yeast Cell Cycle Data FDA View
                                                    • Yeast Cell Cycle Data FDA View (2)
                                                    • Yeast Cell Cycle Data FDA View (3)
                                                    • Yeast Cell Cycles Freq 2 Proj
                                                    • Frequency 2 Analysis
                                                    • Frequency 2 Analysis (2)
                                                    • Frequency 2 Analysis (3)
                                                    • Yeast Cell Cycle
                                                    • SiZer Study of Distrsquon of Angles
                                                    • Reclassification of Major Genes
                                                    • Compare to Previous Classifrsquon
                                                    • New Subpopulation View
                                                    • New Subpopulation View (2)
                                                    • Detailed Look at PCA
                                                    • PCA Rediscovery ndash Renaming
                                                    • PCA Rediscovery ndash Renaming (2)
                                                    • PCA Rediscovery ndash Renaming (3)
                                                    • PCA Rediscovery ndash Renaming (4)
                                                    • PCA Rediscovery ndash Renaming (5)
                                                    • An Interesting Historical Note
                                                    • An Interesting Historical Note (2)
                                                    • Detailed Look at PCA (2)
                                                    • Detailed Look at PCA (3)
                                                    • Review of Linear Algebra
                                                    • Review of Linear Algebra (2)
                                                    • Review of Linear Algebra (3)
                                                    • Review of Linear Algebra (Cont)
                                                    • Review of Linear Algebra (Cont) (2)
                                                    • Review of Linear Algebra (Cont) (3)
                                                    • Review of Linear Algebra (Cont) (4)
                                                    • Review of Linear Algebra (Cont) (5)
                                                    • Review of Linear Algebra (Cont) (6)
                                                    • Review of Linear Algebra (Cont) (7)
                                                    • Review of Linear Algebra (Cont) (8)
                                                    • Review of Linear Algebra (Cont) (9)
                                                    • Review of Linear Algebra (Cont) (10)
                                                    • Review of Linear Algebra (Cont) (11)
                                                    • Review of Linear Algebra (Cont) (12)
                                                    • Review of Linear Algebra (Cont) (13)
                                                    • Review of Linear Algebra (Cont) (14)
                                                    • Review of Linear Algebra (Cont) (15)
                                                    • Review of Linear Algebra (Cont) (16)
                                                    • Review of Linear Algebra (Cont) (17)
                                                    • Review of Linear Algebra (Cont) (18)
                                                    • Review of Linear Algebra (Cont) (19)
                                                    • Review of Linear Algebra (Cont) (20)
                                                    • Review of Linear Algebra (Cont) (21)
                                                    • Review of Linear Algebra (Cont) (22)
                                                    • Review of Linear Algebra (Cont) (23)
                                                    • Review of Linear Algebra (Cont) (24)
                                                    • Review of Linear Algebra (Cont) (25)
                                                    • Review of Linear Algebra (Cont) (26)
                                                    • Review of Linear Algebra (Cont) (27)
                                                    • Review of Linear Algebra (Cont) (28)
                                                    • Review of Linear Algebra (Cont) (29)
                                                    • Review of Linear Algebra (Cont) (30)
                                                    • Review of Linear Algebra (Cont) (31)
                                                    • Review of Linear Algebra (Cont) (32)
                                                    • Review of Linear Algebra (Cont) (33)
                                                    • Review of Linear Algebra (Cont) (34)
                                                    • Review of Linear Algebra (Cont) (35)
                                                    • Review of Linear Algebra (Cont) (36)
                                                    • Review of Linear Algebra (Cont) (37)
                                                    • Review of Linear Algebra (Cont) (38)
                                                    • Review of Linear Algebra (Cont) (39)
                                                    • Review of Linear Algebra (Cont) (40)
                                                    • Review of Linear Algebra (Cont) (41)
                                                    • Review of Linear Algebra (Cont) (42)
                                                    • Review of Linear Algebra (Cont) (43)
                                                    • Review of Linear Algebra (Cont) (44)
                                                    • Review of Linear Algebra (Cont) (45)
                                                    • Review of Linear Algebra (Cont) (46)
                                                    • Review of Linear Algebra (Cont) (47)
                                                    • Review of Linear Algebra (Cont) (48)
                                                    • Review of Linear Algebra (Cont) (49)
                                                    • Review of Linear Algebra (Cont) (50)
                                                    • Review of Linear Algebra (Cont) (51)
                                                    • Review of Linear Algebra (Cont) (52)
                                                    • Review of Linear Algebra (Cont) (53)
                                                    • Review of Linear Algebra (Cont) (54)
                                                    • Review of Linear Algebra (Cont) (55)
                                                    • Review of Linear Algebra (Cont) (56)
                                                    • Review of Linear Algebra (Cont) (57)
                                                    • Review of Linear Algebra (Cont) (58)
                                                    • Review of Linear Algebra (Cont) (59)
                                                    • Review of Linear Algebra (Cont) (60)
                                                    • Review of Linear Algebra (Cont) (61)
                                                    • Review of Linear Algebra (Cont) (62)
                                                    • Review of Linear Algebra (Cont) (63)
                                                    • Review of Linear Algebra (Cont) (64)
                                                    • Review of Linear Algebra (Cont) (65)
                                                    • Review of Linear Algebra (Cont) (66)
                                                    • Review of Linear Algebra (Cont) (67)
                                                    • Review of Linear Algebra (Cont) (68)
                                                    • Review of Linear Algebra (Cont) (69)
                                                    • Review of Linear Algebra (Cont) (70)
                                                    • Review of Linear Algebra (Cont) (71)
                                                    • Review of Linear Algebra (Cont) (72)
                                                    • Review of Linear Algebra (Cont) (73)
                                                    • Review of Linear Algebra (Cont) (74)
                                                    • Review of Linear Algebra (Cont) (75)
                                                    • Review of Linear Algebra (Cont) (76)
                                                    • Review of Linear Algebra (Cont) (77)
                                                    • Review of Linear Algebra (Cont) (78)
                                                    • Review of Linear Algebra (Cont) (79)
                                                    • Review of Linear Algebra (Cont) (80)
                                                    • Review of Linear Algebra (Cont) (81)

                                                      Yeast Cell CycleRevisit ldquophase classificationrdquo

                                                      approachbull Use outer 200 genes

                                                      (other numbers tried less resolution)bull Study distribution of anglesbull Use SiZer analysis

                                                      (finds significant bumps etc in histogram)

                                                      bull Carefully redrew boundariesbull Check by studying kde angles

                                                      SiZer Study of Distrsquon of Angles

                                                      Reclassification of Major Genes

                                                      Compare to Previous Classifrsquon

                                                      New Subpopulation View

                                                      New Subpopulation View

                                                      NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                      Detailed Look at PCA

                                                      Now Study ldquoFolklorerdquo More Carefully

                                                      bull BackGround

                                                      bull History

                                                      bull Underpinnings

                                                      (Mathematical amp Computational)

                                                      Good Overall Reference Jolliffe (2002)

                                                      PCA Rediscovery ndash Renaming

                                                      Statistics Principal Component Analysis (PCA)

                                                      PCA Rediscovery ndash Renaming

                                                      Statistics Principal Component Analysis (PCA)

                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                      PCA Rediscovery ndash Renaming

                                                      Statistics Principal Component Analysis (PCA)

                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                      PCA Rediscovery ndash Renaming

                                                      Statistics Principal Component Analysis (PCA)

                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                                      PCA Rediscovery ndash Renaming

                                                      Statistics Principal Component Analysis (PCA)

                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                                      Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                      An Interesting Historical Note

                                                      The 1st () application of PCA to Functional

                                                      Data Analysis

                                                      An Interesting Historical Note

                                                      The 1st () application of PCA to Functional

                                                      Data Analysis

                                                      Rao (1958)

                                                      1st Paper with ldquoCurves as Data Objectsrdquo

                                                      viewpoint

                                                      Detailed Look at PCA

                                                      Three Important (amp Interesting) Viewpoints

                                                      1 Mathematics

                                                      2 Numerics

                                                      3 Statistics

                                                      Goal Study Interrelationships

                                                      Detailed Look at PCA

                                                      Three Important (amp Interesting) Viewpoints

                                                      1 Mathematics

                                                      2 Numerics

                                                      3 Statistics

                                                      1st Review Linear Alg and Multivar Prob

                                                      Review of Linear Algebra

                                                      Vector Space

                                                      bull set of ldquovectorsrdquo

                                                      bull and ldquoscalarsrdquo (coefficients)

                                                      xa

                                                      Review of Linear Algebra

                                                      Vector Space

                                                      bull set of ldquovectorsrdquo

                                                      bull and ldquoscalarsrdquo (coefficients)

                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                      ( in space)

                                                      xa

                                                      i

                                                      ii xa

                                                      Review of Linear Algebra

                                                      Vector Space

                                                      bull set of ldquovectorsrdquo

                                                      bull and ldquoscalarsrdquo (coefficients)

                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                      ( in space)

                                                      eg

                                                      ldquo dim Euclidrsquon spacerdquo

                                                      xa

                                                      i

                                                      ii xa

                                                      d

                                                      d

                                                      d xx

                                                      x

                                                      x

                                                      x 1

                                                      1

                                                      d

                                                      Review of Linear Algebra (Cont)

                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                      Review of Linear Algebra (Cont)

                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                      Review of Linear Algebra (Cont)

                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                      Note Planes not Through the Origin

                                                      are not Subspaces

                                                      (Do not Contain )00 x

                                                      Review of Linear Algebra (Cont)

                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                      (all Linear Combos of them =

                                                      = Containing Hyperplane

                                                      through Origin)

                                                      Review of Linear Algebra (Cont)

                                                      Basis of Subspace Set of Vectors that

                                                      bull Span ie Everything is a Lin Com of them

                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                      Review of Linear Algebra (Cont)

                                                      Basis of Subspace Set of Vectors that

                                                      bull Span ie Everything is a Lin Com of them

                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                      bull eg ldquoUnit Vector Basisrdquo

                                                      bull Since

                                                      d

                                                      1

                                                      0

                                                      0

                                                      0

                                                      1

                                                      0

                                                      0

                                                      0

                                                      1

                                                      1

                                                      0

                                                      0

                                                      0

                                                      1

                                                      0

                                                      0

                                                      0

                                                      1

                                                      212

                                                      1

                                                      d

                                                      d

                                                      xxx

                                                      x

                                                      x

                                                      x

                                                      Review of Linear Algebra (Cont)

                                                      Basis Matrix of subspace of

                                                      Given a basis

                                                      create matrix of columns

                                                      dnvv 1

                                                      nddnd

                                                      n

                                                      n

                                                      vv

                                                      vv

                                                      vvB

                                                      1

                                                      111

                                                      1

                                                      Review of Linear Algebra (Cont)

                                                      Then linear combo is a matrix multiplicatrsquon

                                                      where

                                                      n

                                                      iii aBva

                                                      1

                                                      na

                                                      a

                                                      a 1

                                                      Review of Linear Algebra (Cont)

                                                      Then linear combo is a matrix multiplicatrsquon

                                                      where

                                                      Note Right Multiplication Gives

                                                      Linear Combination of Column Vectors

                                                      n

                                                      iii aBva

                                                      1

                                                      na

                                                      a

                                                      a 1

                                                      Review of Linear Algebra (Cont)

                                                      Then linear combo is a matrix multiplicatrsquon

                                                      where

                                                      Check sizes

                                                      n

                                                      iii aBva

                                                      1

                                                      na

                                                      a

                                                      a 1

                                                      )1()(1 nndd

                                                      Review of Linear Algebra (Cont)

                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                      For matrices

                                                      Define the Matrix Product

                                                      mkk

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      nmm

                                                      n

                                                      bb

                                                      bb

                                                      B

                                                      1

                                                      111

                                                      m

                                                      iniik

                                                      m

                                                      iiik

                                                      m

                                                      inii

                                                      m

                                                      iii

                                                      baba

                                                      baba

                                                      AB

                                                      1

                                                      11

                                                      11

                                                      111

                                                      Review of Linear Algebra (Cont)

                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                      For matrices

                                                      Define the Matrix Product

                                                      (Inner Products of Rows With Columns )

                                                      mkk

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      nmm

                                                      n

                                                      bb

                                                      bb

                                                      B

                                                      1

                                                      111

                                                      m

                                                      iniik

                                                      m

                                                      iiik

                                                      m

                                                      inii

                                                      m

                                                      iii

                                                      baba

                                                      baba

                                                      AB

                                                      1

                                                      11

                                                      11

                                                      111

                                                      A B

                                                      Review of Linear Algebra (Cont)

                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                      For matrices

                                                      Define the Matrix Product

                                                      (Inner Products of Rows With Columns )

                                                      (Composition of Linear Transformations)

                                                      mkk

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      nmm

                                                      n

                                                      bb

                                                      bb

                                                      B

                                                      1

                                                      111

                                                      m

                                                      iniik

                                                      m

                                                      iiik

                                                      m

                                                      inii

                                                      m

                                                      iii

                                                      baba

                                                      baba

                                                      AB

                                                      1

                                                      11

                                                      11

                                                      111

                                                      A B

                                                      Review of Linear Algebra (Cont)

                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                      For matrices

                                                      Define the Matrix Product

                                                      (Inner Products of Rows With Columns )

                                                      (Composition of Linear Transformations)

                                                      Often Useful to Check Sizes

                                                      mkk

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      nmm

                                                      n

                                                      bb

                                                      bb

                                                      B

                                                      1

                                                      111

                                                      m

                                                      iniik

                                                      m

                                                      iiik

                                                      m

                                                      inii

                                                      m

                                                      iii

                                                      baba

                                                      baba

                                                      AB

                                                      1

                                                      11

                                                      11

                                                      111

                                                      nmmknk

                                                      A B

                                                      Review of Linear Algebra (Cont)

                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                      For matrices

                                                      Define the Matrix Product

                                                      (Inner Products of Rows With Columns )

                                                      (Composition of Linear Transformations)

                                                      Often Useful to Check Sizes

                                                      mkk

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      nmm

                                                      n

                                                      bb

                                                      bb

                                                      B

                                                      1

                                                      111

                                                      m

                                                      iniik

                                                      m

                                                      iiik

                                                      m

                                                      inii

                                                      m

                                                      iii

                                                      baba

                                                      baba

                                                      AB

                                                      1

                                                      11

                                                      11

                                                      111

                                                      nmmknk

                                                      A B

                                                      Review of Linear Algebra (Cont)

                                                      Matrix Trace

                                                      bull For a Square Matrix

                                                      bull Define

                                                      mmm

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      m

                                                      iiiaAtr

                                                      1)(

                                                      Review of Linear Algebra (Cont)

                                                      Matrix Trace

                                                      bull For a Square Matrix

                                                      bull Define

                                                      bull Trace Commutes with Matrix Multiplication

                                                      mmm

                                                      m

                                                      aa

                                                      aa

                                                      A

                                                      1

                                                      111

                                                      m

                                                      iiiaAtr

                                                      1)(

                                                      BAtrABtr

                                                      Review of Linear Algebra (Cont)

                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                      bull Number of Elements in a Basis (Unique)

                                                      Review of Linear Algebra (Cont)

                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                      bull Number of Elements in a Basis (Unique)

                                                      bull (Use Basis Above)

                                                      bull eg dim of a line is 1

                                                      bull eg dim of a plane is 2

                                                      dd dim

                                                      Review of Linear Algebra (Cont)

                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                      bull Number of Elements in a Basis (Unique)

                                                      bull (Use Basis Above)

                                                      bull eg dim of a line is 1

                                                      bull eg dim of a plane is 2

                                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                                      (in Statistical Uses eg ANOVA)

                                                      dd dim

                                                      Review of Linear Algebra (Cont)

                                                      Norm of a Vector

                                                      bull in d 21

                                                      21

                                                      1

                                                      2 xxxx td

                                                      jj

                                                      Review of Linear Algebra (Cont)

                                                      Norm of a Vector

                                                      bull in

                                                      bull Idea length of the vector

                                                      d 2121

                                                      1

                                                      2 xxxx td

                                                      jj

                                                      Review of Linear Algebra (Cont)

                                                      Norm of a Vector

                                                      bull in

                                                      bull Idea length of the vector

                                                      bull Note strange properties for high

                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                      d 2121

                                                      1

                                                      2 xxxx td

                                                      jj

                                                      d

                                                      d

                                                      Review of Linear Algebra (Cont)

                                                      Norm of a Vector (cont)

                                                      bull Length Normalized Vector

                                                      (has Length 1 thus on Surf of Unit Sphere

                                                      amp is a Direction Vector)

                                                      x

                                                      x

                                                      Review of Linear Algebra (Cont)

                                                      Norm of a Vector (cont)

                                                      bull Length Normalized Vector

                                                      (has Length 1 thus on Surf of Unit Sphere

                                                      amp is a Direction Vector)

                                                      bull Define Distance as

                                                      x

                                                      x

                                                      yxyxyxyxd t

                                                      Review of Linear Algebra (Cont)

                                                      Inner (Dot Scalar) Product

                                                      bull for Vectors and

                                                      yxyxyx td

                                                      jjj

                                                      1

                                                      x y

                                                      Review of Linear Algebra (Cont)

                                                      Inner (Dot Scalar) Product

                                                      bull for Vectors and

                                                      bull Related to Norm via

                                                      yxyxyx td

                                                      jjj

                                                      1

                                                      x y

                                                      21 xxx

                                                      Review of Linear Algebra (Cont)

                                                      Inner (Dot Scalar) Product (cont)

                                                      bull measures ldquoangle between and rdquo as

                                                      yyxx

                                                      yx

                                                      yx

                                                      yxyxangle

                                                      tt

                                                      t

                                                      11 cos

                                                      cos

                                                      x y

                                                      Review of Linear Algebra (Cont)

                                                      Inner (Dot Scalar) Product (cont)

                                                      bull measures ldquoangle between and rdquo as

                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                      if and only if

                                                      yyxx

                                                      yx

                                                      yx

                                                      yxyxangle

                                                      tt

                                                      t

                                                      11 cos

                                                      cos

                                                      x y

                                                      yx 0 yx

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis

                                                      bull All Orthogonal to each other

                                                      ie for

                                                      bull All have Length 1

                                                      ie for

                                                      nvv 1

                                                      1 ii vv

                                                      0 ii vv ii

                                                      ni 1

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis (cont)

                                                      bull Spectral Representation

                                                      where

                                                      nvv 1

                                                      n

                                                      iii vax

                                                      1

                                                      ii vxa

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis (cont)

                                                      bull Spectral Representation

                                                      where

                                                      (Coefficient is Inner Product Cool Notation)

                                                      nvv 1

                                                      n

                                                      iii vax

                                                      1

                                                      ii vxa

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis (cont)

                                                      bull Spectral Representation

                                                      where

                                                      Check

                                                      nvv 1

                                                      n

                                                      iii vax

                                                      1

                                                      ii vxa

                                                      iii

                                                      n

                                                      iii

                                                      n

                                                      iiii avvavvavx

                                                      1

                                                      1

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis (cont)

                                                      bull Spectral Representation

                                                      where

                                                      Check

                                                      bull Matrix Notation where ie

                                                      For the Basis Matrix

                                                      nvv 1

                                                      n

                                                      iii vax

                                                      1

                                                      ii vxa

                                                      iii

                                                      n

                                                      iii

                                                      n

                                                      iiii avvavvavx

                                                      1

                                                      1

                                                      aBx Bxa tt xBa t

                                                      nvvB 1

                                                      Review of Linear Algebra (Cont)

                                                      Orthonormal Basis (cont)

                                                      bull Spectral Representation

                                                      where

                                                      Check

                                                      bull Matrix Notation where ie

                                                      is called transform of

                                                      (eg Fourier or Wavelet)

                                                      nvv 1

                                                      n

                                                      iii vax

                                                      1

                                                      ii vxa

                                                      iii

                                                      n

                                                      iii

                                                      n

                                                      iiii avvavvavx

                                                      1

                                                      1

                                                      aBx Bxa tt xBa t

                                                      xa

                                                      Review of Linear Algebra (Cont)

                                                      Parseval identity for

                                                      in subsp genrsquod by o n basis

                                                      x

                                                      nvv 1

                                                      2

                                                      1

                                                      22

                                                      1

                                                      2 aavxx

                                                      n

                                                      ii

                                                      n

                                                      ii

                                                      Review of Linear Algebra (Cont)

                                                      Parseval identity for

                                                      in subsp genrsquod by o n basis

                                                      bull Pythagorean theorem

                                                      bull ldquoDecomposition of Energyrdquo

                                                      bull ANOVA - sums of squares

                                                      x

                                                      nvv 1

                                                      2

                                                      1

                                                      22

                                                      1

                                                      2 aavxx

                                                      n

                                                      ii

                                                      n

                                                      ii

                                                      Review of Linear Algebra (Cont)

                                                      Parseval identity for

                                                      in subsp genrsquod by o n basis

                                                      bull Pythagorean theorem

                                                      bull ldquoDecomposition of Energyrdquo

                                                      bull ANOVA - sums of squares

                                                      bull Transform has same length as

                                                      ie ldquorotation in rdquo

                                                      x

                                                      nvv 1

                                                      2

                                                      1

                                                      22

                                                      1

                                                      2 aavxx

                                                      n

                                                      ii

                                                      n

                                                      ii

                                                      a xd

                                                      Projection of a Vector onto a Subspace

                                                      bull Idea Member of that is Closest to

                                                      (ie ldquoBest Approxrsquonrdquo)

                                                      Review of Linear Algebra (Cont)x

                                                      xV

                                                      V

                                                      Projection of a Vector onto a Subspace

                                                      bull Idea Member of that is Closest to

                                                      (ie ldquoBest Approxrsquonrdquo)

                                                      bull Find that Solves

                                                      (ldquoLeast Squaresrdquo)

                                                      Review of Linear Algebra (Cont)x

                                                      xV

                                                      V

                                                      VxPV vxVv

                                                      min

                                                      Projection of a Vector onto a Subspace

                                                      bull Idea Member of that is Closest to

                                                      (ie ldquoBest Approxrsquonrdquo)

                                                      bull Find that Solves

                                                      (ldquoLeast Squaresrdquo)

                                                      bull For Inner Product (Hilbert) Space

                                                      Exists and is Unique

                                                      Review of Linear Algebra (Cont)x

                                                      xV

                                                      V

                                                      VxPV vxVv

                                                      min

                                                      xPV

                                                      Projection of a Vector onto a Subspace (cont)

                                                      bull General Solution in for Basis Matrix

                                                      Review of Linear Algebra (Cont)

                                                      d VB

                                                      xBBBBxP tVV

                                                      tVVV

                                                      1

                                                      Projection of a Vector onto a Subspace (cont)

                                                      bull General Solution in for Basis Matrix

                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                      (thus projection is another linear operation)

                                                      Review of Linear Algebra (Cont)

                                                      d VB

                                                      xBBBBxP tVV

                                                      tVVV

                                                      1

                                                      tVV

                                                      tVVV BBBBP

                                                      1

                                                      Projection of a Vector onto a Subspace (cont)

                                                      bull General Solution in for Basis Matrix

                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                      (thus projection is another linear operation)

                                                      (note same operation underlies least squares)

                                                      Review of Linear Algebra (Cont)

                                                      d VB

                                                      xBBBBxP tVV

                                                      tVVV

                                                      1

                                                      tVV

                                                      tVVV BBBBP

                                                      1

                                                      Review of Linear Algebra (Cont)

                                                      Projection using Orthonormal Basis

                                                      bull Basis Matrix is Orthonormal

                                                      nnVtV IBB

                                                      10

                                                      01

                                                      1

                                                      111

                                                      1

                                                      1

                                                      nnn

                                                      n

                                                      ntn

                                                      t

                                                      vvvv

                                                      vvvv

                                                      vv

                                                      v

                                                      v

                                                      nvv 1

                                                      Review of Linear Algebra (Cont)

                                                      Projection using Orthonormal Basis

                                                      bull Basis Matrix is Orthonormal

                                                      bull So =

                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                      (Recall Right Multrsquon)

                                                      nnVtV IBB

                                                      10

                                                      01

                                                      1

                                                      111

                                                      1

                                                      1

                                                      nnn

                                                      n

                                                      ntn

                                                      t

                                                      vvvv

                                                      vvvv

                                                      vv

                                                      v

                                                      v

                                                      xBBxP tVVV

                                                      x V

                                                      nvv 1

                                                      Review of Linear Algebra (Cont)

                                                      Projection using Orthonormal Basis (cont)

                                                      bull For Orthogonal Complement

                                                      and

                                                      V

                                                      xPxPx VV 222xPxPx VV

                                                      Review of Linear Algebra (Cont)

                                                      Projection using Orthonormal Basis (cont)

                                                      bull For Orthogonal Complement

                                                      and

                                                      bull Parseval Inequality

                                                      V

                                                      xPxPx VV 222xPxPx VV

                                                      2

                                                      1

                                                      22

                                                      1

                                                      22 aavxxxP

                                                      n

                                                      ii

                                                      n

                                                      iiV

                                                      Review of Linear Algebra (Cont)

                                                      (Real) Unitary Matrices with

                                                      bull Orthonormal Basis Matrix

                                                      (So All of Above Applies)

                                                      ddU IUU t

                                                      Review of Linear Algebra (Cont)

                                                      (Real) Unitary Matrices with

                                                      bull Orthonormal Basis Matrix

                                                      (So All of Above Applies)

                                                      bull Note Transformrsquon is Distance Preserving

                                                      ddU IUU t

                                                      yxdyxyxyxUyUxUdn

                                                      i ii 2

                                                      1

                                                      Review of Linear Algebra (Cont)

                                                      (Real) Unitary Matrices with

                                                      bull Orthonormal Basis Matrix

                                                      (So All of Above Applies)

                                                      bull Note Transformrsquon is Distance Preserving

                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                      ddU IUU t

                                                      yxdyxyxyxUyUxUdn

                                                      i ii 2

                                                      1

                                                      U

                                                      Review of Linear Algebra (Cont)

                                                      Singular Value Decomposition (SVD)

                                                      For a Matrix

                                                      Find

                                                      ndX

                                                      Review of Linear Algebra (Cont)

                                                      Singular Value Decomposition (SVD)

                                                      For a Matrix

                                                      Find a Diagonal Matrix

                                                      with Entries

                                                      called Singular Values

                                                      ndX

                                                      ndS

                                                      )min(1 ndss

                                                      Review of Linear Algebra (Cont)

                                                      Singular Value Decomposition (SVD)

                                                      For a Matrix

                                                      Find a Diagonal Matrix

                                                      with Entries

                                                      called Singular Values

                                                      And Unitary (Rotation) Matrices

                                                      (recall )

                                                      ndX

                                                      ndS

                                                      )min(1 ndss

                                                      ddU nnV

                                                      IVVUU tt

                                                      Review of Linear Algebra (Cont)

                                                      Singular Value Decomposition (SVD)

                                                      For a Matrix

                                                      Find a Diagonal Matrix

                                                      with Entries

                                                      called Singular Values

                                                      And Unitary (Rotation) Matrices

                                                      (recall )

                                                      So That

                                                      ndX

                                                      ndS

                                                      )min(1 ndss

                                                      ddU nnV

                                                      IVVUU tt tUSVX

                                                      Review of Linear Algebra (Cont)

                                                      Intuition behind Singular Value Decomposition

                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                      vVSUvVSUvX tt

                                                      Review of Linear Algebra (Cont)

                                                      Intuition behind Singular Value Decomposition

                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                      bull First rotate

                                                      X

                                                      vVSUvVSUvX tt

                                                      Review of Linear Algebra (Cont)

                                                      Intuition behind Singular Value Decomposition

                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                      bull First rotate

                                                      bull Second rescale coordinate axes (by )

                                                      X

                                                      vVSUvVSUvX tt

                                                      is

                                                      Review of Linear Algebra (Cont)

                                                      Intuition behind Singular Value Decomposition

                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                      bull First rotate

                                                      bull Second rescale coordinate axes (by )

                                                      bull Third rotate again

                                                      X

                                                      vVSUvVSUvX tt

                                                      is

                                                      Review of Linear Algebra (Cont)

                                                      Intuition behind Singular Value Decomposition

                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                      bull First rotate

                                                      bull Second rescale coordinate axes (by )

                                                      bull Third rotate again

                                                      bull ie have diagonalized the transformation

                                                      X

                                                      vVSUvVSUvX tt

                                                      is

                                                      SVD Compact Representation

                                                      Useful Labeling

                                                      Singular Values in Increasing Order

                                                      Review of Linear Algebra (Cont)

                                                      )min(1 dnss

                                                      SVD Compact Representation

                                                      Useful Labeling

                                                      Singular Values in Increasing Order

                                                      Note singular values = 0 can be omitted

                                                      (Since do ldquo0-Stretchingrdquo)

                                                      Review of Linear Algebra (Cont)

                                                      )min(1 dnss

                                                      Review of Linear Algebra (Cont)

                                                      r

                                                      SVD Compact Representation

                                                      Useful Labeling

                                                      Singular Values in Increasing Order

                                                      Note singular values = 0 can be omitted

                                                      Let = of positive singular values

                                                      )min(1 dnss

                                                      Review of Linear Algebra (Cont)

                                                      r

                                                      SVD Compact Representation

                                                      Useful Labeling

                                                      Singular Values in Increasing Order

                                                      Note singular values = 0 can be omitted

                                                      Let = of positive singular values

                                                      Then

                                                      Where are truncations of

                                                      trnrrrd VSUX

                                                      VSU

                                                      )min(1 dnss

                                                      Review of Linear Algebra (Cont)

                                                      SVD Full Representation

                                                      =

                                                      Graphics Display Assumes

                                                      ndX ddU ndS nn

                                                      tV

                                                      nd

                                                      Review of Linear Algebra (Cont)

                                                      SVD Full Representation

                                                      =

                                                      Full Rank Basis Matrix

                                                      ndX ddU ndS nn

                                                      tV

                                                      Review of Linear Algebra (Cont)

                                                      SVD Full Representation

                                                      =

                                                      Full Rank Basis Matrix

                                                      All 0s in Bottom

                                                      ndX ddU ndS nn

                                                      tV

                                                      Review of Linear Algebra (Cont)

                                                      SVD Reduced Representation

                                                      =

                                                      These Columns Get 0ed Out

                                                      ndX ddU nnS nn

                                                      tV

                                                      nnd 0

                                                      Review of Linear Algebra (Cont)

                                                      SVD Reduced Representation

                                                      =ndX ndU nnS nn

                                                      tV

                                                      Review of Linear Algebra (Cont)

                                                      SVD Reduced Representation

                                                      =

                                                      Also Some of These May be 0

                                                      ndX ndU nnS nn

                                                      tV

                                                      Review of Linear Algebra (Cont)

                                                      SVD Compact Representation

                                                      = ndX rdU

                                                      rrS nrtV

                                                      0

                                                      Review of Linear Algebra (Cont)

                                                      SVD Compact Representation

                                                      =

                                                      These Get 0ed Out

                                                      ndX rdU

                                                      rrS nrtV

                                                      0

                                                      Review of Linear Algebra (Cont)

                                                      SVD Compact Representation

                                                      = ndX rdU

                                                      rrS nrtV

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition

                                                      For a (Symmetric) Square Matrix

                                                      Find

                                                      ddX

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition

                                                      For a (Symmetric) Square Matrix

                                                      Find a Diagonal Matrix

                                                      Called Eigenvalues

                                                      Convenient Ordering

                                                      ddX

                                                      d

                                                      D

                                                      0

                                                      01

                                                      n 1

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition

                                                      For a (Symmetric) Square Matrix

                                                      Find a Diagonal Matrix

                                                      And an Orthonormal Matrix

                                                      (ie )

                                                      ddX

                                                      d

                                                      D

                                                      0

                                                      01

                                                      ddB

                                                      ddtt IBBBB

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition

                                                      For a (Symmetric) Square Matrix

                                                      Find a Diagonal Matrix

                                                      And an Orthonormal Matrix

                                                      (ie )

                                                      So that ie

                                                      ddX

                                                      d

                                                      D

                                                      0

                                                      01

                                                      ddB

                                                      ddtt IBBBB

                                                      DBBX tBDBX

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                      (looks similar)

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                      Complex (uses )

                                                      VU

                                                      1i

                                                      Review of Linear Algebra (Cont)

                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                      VU

                                                      X

                                                      BVU

                                                      1i

                                                      • Participant Presentations
                                                      • PCA to find clusters
                                                      • Statistical Smoothing
                                                      • Kernel Density Estimation
                                                      • Scatterplot Smoothing
                                                      • Statistical Smoothing (2)
                                                      • SiZer Background
                                                      • SiZer Background (2)
                                                      • SiZer Background (3)
                                                      • SiZer Background (4)
                                                      • SiZer Background (5)
                                                      • SiZer Overview
                                                      • PCA to find clusters (2)
                                                      • PCA to find clusters (3)
                                                      • PCA to find clusters (4)
                                                      • PCA to find clusters (5)
                                                      • PCA to find clusters (6)
                                                      • PCA to find clusters (7)
                                                      • Recall Yeast Cell Cycle Data
                                                      • Yeast Cell Cycle Data FDA View
                                                      • Yeast Cell Cycle Data FDA View (2)
                                                      • Yeast Cell Cycle Data FDA View (3)
                                                      • Yeast Cell Cycles Freq 2 Proj
                                                      • Frequency 2 Analysis
                                                      • Frequency 2 Analysis (2)
                                                      • Frequency 2 Analysis (3)
                                                      • Yeast Cell Cycle
                                                      • SiZer Study of Distrsquon of Angles
                                                      • Reclassification of Major Genes
                                                      • Compare to Previous Classifrsquon
                                                      • New Subpopulation View
                                                      • New Subpopulation View (2)
                                                      • Detailed Look at PCA
                                                      • PCA Rediscovery ndash Renaming
                                                      • PCA Rediscovery ndash Renaming (2)
                                                      • PCA Rediscovery ndash Renaming (3)
                                                      • PCA Rediscovery ndash Renaming (4)
                                                      • PCA Rediscovery ndash Renaming (5)
                                                      • An Interesting Historical Note
                                                      • An Interesting Historical Note (2)
                                                      • Detailed Look at PCA (2)
                                                      • Detailed Look at PCA (3)
                                                      • Review of Linear Algebra
                                                      • Review of Linear Algebra (2)
                                                      • Review of Linear Algebra (3)
                                                      • Review of Linear Algebra (Cont)
                                                      • Review of Linear Algebra (Cont) (2)
                                                      • Review of Linear Algebra (Cont) (3)
                                                      • Review of Linear Algebra (Cont) (4)
                                                      • Review of Linear Algebra (Cont) (5)
                                                      • Review of Linear Algebra (Cont) (6)
                                                      • Review of Linear Algebra (Cont) (7)
                                                      • Review of Linear Algebra (Cont) (8)
                                                      • Review of Linear Algebra (Cont) (9)
                                                      • Review of Linear Algebra (Cont) (10)
                                                      • Review of Linear Algebra (Cont) (11)
                                                      • Review of Linear Algebra (Cont) (12)
                                                      • Review of Linear Algebra (Cont) (13)
                                                      • Review of Linear Algebra (Cont) (14)
                                                      • Review of Linear Algebra (Cont) (15)
                                                      • Review of Linear Algebra (Cont) (16)
                                                      • Review of Linear Algebra (Cont) (17)
                                                      • Review of Linear Algebra (Cont) (18)
                                                      • Review of Linear Algebra (Cont) (19)
                                                      • Review of Linear Algebra (Cont) (20)
                                                      • Review of Linear Algebra (Cont) (21)
                                                      • Review of Linear Algebra (Cont) (22)
                                                      • Review of Linear Algebra (Cont) (23)
                                                      • Review of Linear Algebra (Cont) (24)
                                                      • Review of Linear Algebra (Cont) (25)
                                                      • Review of Linear Algebra (Cont) (26)
                                                      • Review of Linear Algebra (Cont) (27)
                                                      • Review of Linear Algebra (Cont) (28)
                                                      • Review of Linear Algebra (Cont) (29)
                                                      • Review of Linear Algebra (Cont) (30)
                                                      • Review of Linear Algebra (Cont) (31)
                                                      • Review of Linear Algebra (Cont) (32)
                                                      • Review of Linear Algebra (Cont) (33)
                                                      • Review of Linear Algebra (Cont) (34)
                                                      • Review of Linear Algebra (Cont) (35)
                                                      • Review of Linear Algebra (Cont) (36)
                                                      • Review of Linear Algebra (Cont) (37)
                                                      • Review of Linear Algebra (Cont) (38)
                                                      • Review of Linear Algebra (Cont) (39)
                                                      • Review of Linear Algebra (Cont) (40)
                                                      • Review of Linear Algebra (Cont) (41)
                                                      • Review of Linear Algebra (Cont) (42)
                                                      • Review of Linear Algebra (Cont) (43)
                                                      • Review of Linear Algebra (Cont) (44)
                                                      • Review of Linear Algebra (Cont) (45)
                                                      • Review of Linear Algebra (Cont) (46)
                                                      • Review of Linear Algebra (Cont) (47)
                                                      • Review of Linear Algebra (Cont) (48)
                                                      • Review of Linear Algebra (Cont) (49)
                                                      • Review of Linear Algebra (Cont) (50)
                                                      • Review of Linear Algebra (Cont) (51)
                                                      • Review of Linear Algebra (Cont) (52)
                                                      • Review of Linear Algebra (Cont) (53)
                                                      • Review of Linear Algebra (Cont) (54)
                                                      • Review of Linear Algebra (Cont) (55)
                                                      • Review of Linear Algebra (Cont) (56)
                                                      • Review of Linear Algebra (Cont) (57)
                                                      • Review of Linear Algebra (Cont) (58)
                                                      • Review of Linear Algebra (Cont) (59)
                                                      • Review of Linear Algebra (Cont) (60)
                                                      • Review of Linear Algebra (Cont) (61)
                                                      • Review of Linear Algebra (Cont) (62)
                                                      • Review of Linear Algebra (Cont) (63)
                                                      • Review of Linear Algebra (Cont) (64)
                                                      • Review of Linear Algebra (Cont) (65)
                                                      • Review of Linear Algebra (Cont) (66)
                                                      • Review of Linear Algebra (Cont) (67)
                                                      • Review of Linear Algebra (Cont) (68)
                                                      • Review of Linear Algebra (Cont) (69)
                                                      • Review of Linear Algebra (Cont) (70)
                                                      • Review of Linear Algebra (Cont) (71)
                                                      • Review of Linear Algebra (Cont) (72)
                                                      • Review of Linear Algebra (Cont) (73)
                                                      • Review of Linear Algebra (Cont) (74)
                                                      • Review of Linear Algebra (Cont) (75)
                                                      • Review of Linear Algebra (Cont) (76)
                                                      • Review of Linear Algebra (Cont) (77)
                                                      • Review of Linear Algebra (Cont) (78)
                                                      • Review of Linear Algebra (Cont) (79)
                                                      • Review of Linear Algebra (Cont) (80)
                                                      • Review of Linear Algebra (Cont) (81)

                                                        SiZer Study of Distrsquon of Angles

                                                        Reclassification of Major Genes

                                                        Compare to Previous Classifrsquon

                                                        New Subpopulation View

                                                        New Subpopulation View

                                                        NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                        Detailed Look at PCA

                                                        Now Study ldquoFolklorerdquo More Carefully

                                                        bull BackGround

                                                        bull History

                                                        bull Underpinnings

                                                        (Mathematical amp Computational)

                                                        Good Overall Reference Jolliffe (2002)

                                                        PCA Rediscovery ndash Renaming

                                                        Statistics Principal Component Analysis (PCA)

                                                        PCA Rediscovery ndash Renaming

                                                        Statistics Principal Component Analysis (PCA)

                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                        PCA Rediscovery ndash Renaming

                                                        Statistics Principal Component Analysis (PCA)

                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                        PCA Rediscovery ndash Renaming

                                                        Statistics Principal Component Analysis (PCA)

                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                                        PCA Rediscovery ndash Renaming

                                                        Statistics Principal Component Analysis (PCA)

                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                                        Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                        An Interesting Historical Note

                                                        The 1st () application of PCA to Functional

                                                        Data Analysis

                                                        An Interesting Historical Note

                                                        The 1st () application of PCA to Functional

                                                        Data Analysis

                                                        Rao (1958)

                                                        1st Paper with ldquoCurves as Data Objectsrdquo

                                                        viewpoint

                                                        Detailed Look at PCA

                                                        Three Important (amp Interesting) Viewpoints

                                                        1 Mathematics

                                                        2 Numerics

                                                        3 Statistics

                                                        Goal Study Interrelationships

                                                        Detailed Look at PCA

                                                        Three Important (amp Interesting) Viewpoints

                                                        1 Mathematics

                                                        2 Numerics

                                                        3 Statistics

                                                        1st Review Linear Alg and Multivar Prob

                                                        Review of Linear Algebra

                                                        Vector Space

                                                        bull set of ldquovectorsrdquo

                                                        bull and ldquoscalarsrdquo (coefficients)

                                                        xa

                                                        Review of Linear Algebra

                                                        Vector Space

                                                        bull set of ldquovectorsrdquo

                                                        bull and ldquoscalarsrdquo (coefficients)

                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                        ( in space)

                                                        xa

                                                        i

                                                        ii xa

                                                        Review of Linear Algebra

                                                        Vector Space

                                                        bull set of ldquovectorsrdquo

                                                        bull and ldquoscalarsrdquo (coefficients)

                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                        ( in space)

                                                        eg

                                                        ldquo dim Euclidrsquon spacerdquo

                                                        xa

                                                        i

                                                        ii xa

                                                        d

                                                        d

                                                        d xx

                                                        x

                                                        x

                                                        x 1

                                                        1

                                                        d

                                                        Review of Linear Algebra (Cont)

                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                        Review of Linear Algebra (Cont)

                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                        Review of Linear Algebra (Cont)

                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                        Note Planes not Through the Origin

                                                        are not Subspaces

                                                        (Do not Contain )00 x

                                                        Review of Linear Algebra (Cont)

                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                        (all Linear Combos of them =

                                                        = Containing Hyperplane

                                                        through Origin)

                                                        Review of Linear Algebra (Cont)

                                                        Basis of Subspace Set of Vectors that

                                                        bull Span ie Everything is a Lin Com of them

                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                        Review of Linear Algebra (Cont)

                                                        Basis of Subspace Set of Vectors that

                                                        bull Span ie Everything is a Lin Com of them

                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                        bull eg ldquoUnit Vector Basisrdquo

                                                        bull Since

                                                        d

                                                        1

                                                        0

                                                        0

                                                        0

                                                        1

                                                        0

                                                        0

                                                        0

                                                        1

                                                        1

                                                        0

                                                        0

                                                        0

                                                        1

                                                        0

                                                        0

                                                        0

                                                        1

                                                        212

                                                        1

                                                        d

                                                        d

                                                        xxx

                                                        x

                                                        x

                                                        x

                                                        Review of Linear Algebra (Cont)

                                                        Basis Matrix of subspace of

                                                        Given a basis

                                                        create matrix of columns

                                                        dnvv 1

                                                        nddnd

                                                        n

                                                        n

                                                        vv

                                                        vv

                                                        vvB

                                                        1

                                                        111

                                                        1

                                                        Review of Linear Algebra (Cont)

                                                        Then linear combo is a matrix multiplicatrsquon

                                                        where

                                                        n

                                                        iii aBva

                                                        1

                                                        na

                                                        a

                                                        a 1

                                                        Review of Linear Algebra (Cont)

                                                        Then linear combo is a matrix multiplicatrsquon

                                                        where

                                                        Note Right Multiplication Gives

                                                        Linear Combination of Column Vectors

                                                        n

                                                        iii aBva

                                                        1

                                                        na

                                                        a

                                                        a 1

                                                        Review of Linear Algebra (Cont)

                                                        Then linear combo is a matrix multiplicatrsquon

                                                        where

                                                        Check sizes

                                                        n

                                                        iii aBva

                                                        1

                                                        na

                                                        a

                                                        a 1

                                                        )1()(1 nndd

                                                        Review of Linear Algebra (Cont)

                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                        For matrices

                                                        Define the Matrix Product

                                                        mkk

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        nmm

                                                        n

                                                        bb

                                                        bb

                                                        B

                                                        1

                                                        111

                                                        m

                                                        iniik

                                                        m

                                                        iiik

                                                        m

                                                        inii

                                                        m

                                                        iii

                                                        baba

                                                        baba

                                                        AB

                                                        1

                                                        11

                                                        11

                                                        111

                                                        Review of Linear Algebra (Cont)

                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                        For matrices

                                                        Define the Matrix Product

                                                        (Inner Products of Rows With Columns )

                                                        mkk

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        nmm

                                                        n

                                                        bb

                                                        bb

                                                        B

                                                        1

                                                        111

                                                        m

                                                        iniik

                                                        m

                                                        iiik

                                                        m

                                                        inii

                                                        m

                                                        iii

                                                        baba

                                                        baba

                                                        AB

                                                        1

                                                        11

                                                        11

                                                        111

                                                        A B

                                                        Review of Linear Algebra (Cont)

                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                        For matrices

                                                        Define the Matrix Product

                                                        (Inner Products of Rows With Columns )

                                                        (Composition of Linear Transformations)

                                                        mkk

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        nmm

                                                        n

                                                        bb

                                                        bb

                                                        B

                                                        1

                                                        111

                                                        m

                                                        iniik

                                                        m

                                                        iiik

                                                        m

                                                        inii

                                                        m

                                                        iii

                                                        baba

                                                        baba

                                                        AB

                                                        1

                                                        11

                                                        11

                                                        111

                                                        A B

                                                        Review of Linear Algebra (Cont)

                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                        For matrices

                                                        Define the Matrix Product

                                                        (Inner Products of Rows With Columns )

                                                        (Composition of Linear Transformations)

                                                        Often Useful to Check Sizes

                                                        mkk

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        nmm

                                                        n

                                                        bb

                                                        bb

                                                        B

                                                        1

                                                        111

                                                        m

                                                        iniik

                                                        m

                                                        iiik

                                                        m

                                                        inii

                                                        m

                                                        iii

                                                        baba

                                                        baba

                                                        AB

                                                        1

                                                        11

                                                        11

                                                        111

                                                        nmmknk

                                                        A B

                                                        Review of Linear Algebra (Cont)

                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                        For matrices

                                                        Define the Matrix Product

                                                        (Inner Products of Rows With Columns )

                                                        (Composition of Linear Transformations)

                                                        Often Useful to Check Sizes

                                                        mkk

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        nmm

                                                        n

                                                        bb

                                                        bb

                                                        B

                                                        1

                                                        111

                                                        m

                                                        iniik

                                                        m

                                                        iiik

                                                        m

                                                        inii

                                                        m

                                                        iii

                                                        baba

                                                        baba

                                                        AB

                                                        1

                                                        11

                                                        11

                                                        111

                                                        nmmknk

                                                        A B

                                                        Review of Linear Algebra (Cont)

                                                        Matrix Trace

                                                        bull For a Square Matrix

                                                        bull Define

                                                        mmm

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        m

                                                        iiiaAtr

                                                        1)(

                                                        Review of Linear Algebra (Cont)

                                                        Matrix Trace

                                                        bull For a Square Matrix

                                                        bull Define

                                                        bull Trace Commutes with Matrix Multiplication

                                                        mmm

                                                        m

                                                        aa

                                                        aa

                                                        A

                                                        1

                                                        111

                                                        m

                                                        iiiaAtr

                                                        1)(

                                                        BAtrABtr

                                                        Review of Linear Algebra (Cont)

                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                        bull Number of Elements in a Basis (Unique)

                                                        Review of Linear Algebra (Cont)

                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                        bull Number of Elements in a Basis (Unique)

                                                        bull (Use Basis Above)

                                                        bull eg dim of a line is 1

                                                        bull eg dim of a plane is 2

                                                        dd dim

                                                        Review of Linear Algebra (Cont)

                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                        bull Number of Elements in a Basis (Unique)

                                                        bull (Use Basis Above)

                                                        bull eg dim of a line is 1

                                                        bull eg dim of a plane is 2

                                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                                        (in Statistical Uses eg ANOVA)

                                                        dd dim

                                                        Review of Linear Algebra (Cont)

                                                        Norm of a Vector

                                                        bull in d 21

                                                        21

                                                        1

                                                        2 xxxx td

                                                        jj

                                                        Review of Linear Algebra (Cont)

                                                        Norm of a Vector

                                                        bull in

                                                        bull Idea length of the vector

                                                        d 2121

                                                        1

                                                        2 xxxx td

                                                        jj

                                                        Review of Linear Algebra (Cont)

                                                        Norm of a Vector

                                                        bull in

                                                        bull Idea length of the vector

                                                        bull Note strange properties for high

                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                        d 2121

                                                        1

                                                        2 xxxx td

                                                        jj

                                                        d

                                                        d

                                                        Review of Linear Algebra (Cont)

                                                        Norm of a Vector (cont)

                                                        bull Length Normalized Vector

                                                        (has Length 1 thus on Surf of Unit Sphere

                                                        amp is a Direction Vector)

                                                        x

                                                        x

                                                        Review of Linear Algebra (Cont)

                                                        Norm of a Vector (cont)

                                                        bull Length Normalized Vector

                                                        (has Length 1 thus on Surf of Unit Sphere

                                                        amp is a Direction Vector)

                                                        bull Define Distance as

                                                        x

                                                        x

                                                        yxyxyxyxd t

                                                        Review of Linear Algebra (Cont)

                                                        Inner (Dot Scalar) Product

                                                        bull for Vectors and

                                                        yxyxyx td

                                                        jjj

                                                        1

                                                        x y

                                                        Review of Linear Algebra (Cont)

                                                        Inner (Dot Scalar) Product

                                                        bull for Vectors and

                                                        bull Related to Norm via

                                                        yxyxyx td

                                                        jjj

                                                        1

                                                        x y

                                                        21 xxx

                                                        Review of Linear Algebra (Cont)

                                                        Inner (Dot Scalar) Product (cont)

                                                        bull measures ldquoangle between and rdquo as

                                                        yyxx

                                                        yx

                                                        yx

                                                        yxyxangle

                                                        tt

                                                        t

                                                        11 cos

                                                        cos

                                                        x y

                                                        Review of Linear Algebra (Cont)

                                                        Inner (Dot Scalar) Product (cont)

                                                        bull measures ldquoangle between and rdquo as

                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                        if and only if

                                                        yyxx

                                                        yx

                                                        yx

                                                        yxyxangle

                                                        tt

                                                        t

                                                        11 cos

                                                        cos

                                                        x y

                                                        yx 0 yx

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis

                                                        bull All Orthogonal to each other

                                                        ie for

                                                        bull All have Length 1

                                                        ie for

                                                        nvv 1

                                                        1 ii vv

                                                        0 ii vv ii

                                                        ni 1

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis (cont)

                                                        bull Spectral Representation

                                                        where

                                                        nvv 1

                                                        n

                                                        iii vax

                                                        1

                                                        ii vxa

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis (cont)

                                                        bull Spectral Representation

                                                        where

                                                        (Coefficient is Inner Product Cool Notation)

                                                        nvv 1

                                                        n

                                                        iii vax

                                                        1

                                                        ii vxa

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis (cont)

                                                        bull Spectral Representation

                                                        where

                                                        Check

                                                        nvv 1

                                                        n

                                                        iii vax

                                                        1

                                                        ii vxa

                                                        iii

                                                        n

                                                        iii

                                                        n

                                                        iiii avvavvavx

                                                        1

                                                        1

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis (cont)

                                                        bull Spectral Representation

                                                        where

                                                        Check

                                                        bull Matrix Notation where ie

                                                        For the Basis Matrix

                                                        nvv 1

                                                        n

                                                        iii vax

                                                        1

                                                        ii vxa

                                                        iii

                                                        n

                                                        iii

                                                        n

                                                        iiii avvavvavx

                                                        1

                                                        1

                                                        aBx Bxa tt xBa t

                                                        nvvB 1

                                                        Review of Linear Algebra (Cont)

                                                        Orthonormal Basis (cont)

                                                        bull Spectral Representation

                                                        where

                                                        Check

                                                        bull Matrix Notation where ie

                                                        is called transform of

                                                        (eg Fourier or Wavelet)

                                                        nvv 1

                                                        n

                                                        iii vax

                                                        1

                                                        ii vxa

                                                        iii

                                                        n

                                                        iii

                                                        n

                                                        iiii avvavvavx

                                                        1

                                                        1

                                                        aBx Bxa tt xBa t

                                                        xa

                                                        Review of Linear Algebra (Cont)

                                                        Parseval identity for

                                                        in subsp genrsquod by o n basis

                                                        x

                                                        nvv 1

                                                        2

                                                        1

                                                        22

                                                        1

                                                        2 aavxx

                                                        n

                                                        ii

                                                        n

                                                        ii

                                                        Review of Linear Algebra (Cont)

                                                        Parseval identity for

                                                        in subsp genrsquod by o n basis

                                                        bull Pythagorean theorem

                                                        bull ldquoDecomposition of Energyrdquo

                                                        bull ANOVA - sums of squares

                                                        x

                                                        nvv 1

                                                        2

                                                        1

                                                        22

                                                        1

                                                        2 aavxx

                                                        n

                                                        ii

                                                        n

                                                        ii

                                                        Review of Linear Algebra (Cont)

                                                        Parseval identity for

                                                        in subsp genrsquod by o n basis

                                                        bull Pythagorean theorem

                                                        bull ldquoDecomposition of Energyrdquo

                                                        bull ANOVA - sums of squares

                                                        bull Transform has same length as

                                                        ie ldquorotation in rdquo

                                                        x

                                                        nvv 1

                                                        2

                                                        1

                                                        22

                                                        1

                                                        2 aavxx

                                                        n

                                                        ii

                                                        n

                                                        ii

                                                        a xd

                                                        Projection of a Vector onto a Subspace

                                                        bull Idea Member of that is Closest to

                                                        (ie ldquoBest Approxrsquonrdquo)

                                                        Review of Linear Algebra (Cont)x

                                                        xV

                                                        V

                                                        Projection of a Vector onto a Subspace

                                                        bull Idea Member of that is Closest to

                                                        (ie ldquoBest Approxrsquonrdquo)

                                                        bull Find that Solves

                                                        (ldquoLeast Squaresrdquo)

                                                        Review of Linear Algebra (Cont)x

                                                        xV

                                                        V

                                                        VxPV vxVv

                                                        min

                                                        Projection of a Vector onto a Subspace

                                                        bull Idea Member of that is Closest to

                                                        (ie ldquoBest Approxrsquonrdquo)

                                                        bull Find that Solves

                                                        (ldquoLeast Squaresrdquo)

                                                        bull For Inner Product (Hilbert) Space

                                                        Exists and is Unique

                                                        Review of Linear Algebra (Cont)x

                                                        xV

                                                        V

                                                        VxPV vxVv

                                                        min

                                                        xPV

                                                        Projection of a Vector onto a Subspace (cont)

                                                        bull General Solution in for Basis Matrix

                                                        Review of Linear Algebra (Cont)

                                                        d VB

                                                        xBBBBxP tVV

                                                        tVVV

                                                        1

                                                        Projection of a Vector onto a Subspace (cont)

                                                        bull General Solution in for Basis Matrix

                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                        (thus projection is another linear operation)

                                                        Review of Linear Algebra (Cont)

                                                        d VB

                                                        xBBBBxP tVV

                                                        tVVV

                                                        1

                                                        tVV

                                                        tVVV BBBBP

                                                        1

                                                        Projection of a Vector onto a Subspace (cont)

                                                        bull General Solution in for Basis Matrix

                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                        (thus projection is another linear operation)

                                                        (note same operation underlies least squares)

                                                        Review of Linear Algebra (Cont)

                                                        d VB

                                                        xBBBBxP tVV

                                                        tVVV

                                                        1

                                                        tVV

                                                        tVVV BBBBP

                                                        1

                                                        Review of Linear Algebra (Cont)

                                                        Projection using Orthonormal Basis

                                                        bull Basis Matrix is Orthonormal

                                                        nnVtV IBB

                                                        10

                                                        01

                                                        1

                                                        111

                                                        1

                                                        1

                                                        nnn

                                                        n

                                                        ntn

                                                        t

                                                        vvvv

                                                        vvvv

                                                        vv

                                                        v

                                                        v

                                                        nvv 1

                                                        Review of Linear Algebra (Cont)

                                                        Projection using Orthonormal Basis

                                                        bull Basis Matrix is Orthonormal

                                                        bull So =

                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                        (Recall Right Multrsquon)

                                                        nnVtV IBB

                                                        10

                                                        01

                                                        1

                                                        111

                                                        1

                                                        1

                                                        nnn

                                                        n

                                                        ntn

                                                        t

                                                        vvvv

                                                        vvvv

                                                        vv

                                                        v

                                                        v

                                                        xBBxP tVVV

                                                        x V

                                                        nvv 1

                                                        Review of Linear Algebra (Cont)

                                                        Projection using Orthonormal Basis (cont)

                                                        bull For Orthogonal Complement

                                                        and

                                                        V

                                                        xPxPx VV 222xPxPx VV

                                                        Review of Linear Algebra (Cont)

                                                        Projection using Orthonormal Basis (cont)

                                                        bull For Orthogonal Complement

                                                        and

                                                        bull Parseval Inequality

                                                        V

                                                        xPxPx VV 222xPxPx VV

                                                        2

                                                        1

                                                        22

                                                        1

                                                        22 aavxxxP

                                                        n

                                                        ii

                                                        n

                                                        iiV

                                                        Review of Linear Algebra (Cont)

                                                        (Real) Unitary Matrices with

                                                        bull Orthonormal Basis Matrix

                                                        (So All of Above Applies)

                                                        ddU IUU t

                                                        Review of Linear Algebra (Cont)

                                                        (Real) Unitary Matrices with

                                                        bull Orthonormal Basis Matrix

                                                        (So All of Above Applies)

                                                        bull Note Transformrsquon is Distance Preserving

                                                        ddU IUU t

                                                        yxdyxyxyxUyUxUdn

                                                        i ii 2

                                                        1

                                                        Review of Linear Algebra (Cont)

                                                        (Real) Unitary Matrices with

                                                        bull Orthonormal Basis Matrix

                                                        (So All of Above Applies)

                                                        bull Note Transformrsquon is Distance Preserving

                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                        ddU IUU t

                                                        yxdyxyxyxUyUxUdn

                                                        i ii 2

                                                        1

                                                        U

                                                        Review of Linear Algebra (Cont)

                                                        Singular Value Decomposition (SVD)

                                                        For a Matrix

                                                        Find

                                                        ndX

                                                        Review of Linear Algebra (Cont)

                                                        Singular Value Decomposition (SVD)

                                                        For a Matrix

                                                        Find a Diagonal Matrix

                                                        with Entries

                                                        called Singular Values

                                                        ndX

                                                        ndS

                                                        )min(1 ndss

                                                        Review of Linear Algebra (Cont)

                                                        Singular Value Decomposition (SVD)

                                                        For a Matrix

                                                        Find a Diagonal Matrix

                                                        with Entries

                                                        called Singular Values

                                                        And Unitary (Rotation) Matrices

                                                        (recall )

                                                        ndX

                                                        ndS

                                                        )min(1 ndss

                                                        ddU nnV

                                                        IVVUU tt

                                                        Review of Linear Algebra (Cont)

                                                        Singular Value Decomposition (SVD)

                                                        For a Matrix

                                                        Find a Diagonal Matrix

                                                        with Entries

                                                        called Singular Values

                                                        And Unitary (Rotation) Matrices

                                                        (recall )

                                                        So That

                                                        ndX

                                                        ndS

                                                        )min(1 ndss

                                                        ddU nnV

                                                        IVVUU tt tUSVX

                                                        Review of Linear Algebra (Cont)

                                                        Intuition behind Singular Value Decomposition

                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                        vVSUvVSUvX tt

                                                        Review of Linear Algebra (Cont)

                                                        Intuition behind Singular Value Decomposition

                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                        bull First rotate

                                                        X

                                                        vVSUvVSUvX tt

                                                        Review of Linear Algebra (Cont)

                                                        Intuition behind Singular Value Decomposition

                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                        bull First rotate

                                                        bull Second rescale coordinate axes (by )

                                                        X

                                                        vVSUvVSUvX tt

                                                        is

                                                        Review of Linear Algebra (Cont)

                                                        Intuition behind Singular Value Decomposition

                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                        bull First rotate

                                                        bull Second rescale coordinate axes (by )

                                                        bull Third rotate again

                                                        X

                                                        vVSUvVSUvX tt

                                                        is

                                                        Review of Linear Algebra (Cont)

                                                        Intuition behind Singular Value Decomposition

                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                        bull First rotate

                                                        bull Second rescale coordinate axes (by )

                                                        bull Third rotate again

                                                        bull ie have diagonalized the transformation

                                                        X

                                                        vVSUvVSUvX tt

                                                        is

                                                        SVD Compact Representation

                                                        Useful Labeling

                                                        Singular Values in Increasing Order

                                                        Review of Linear Algebra (Cont)

                                                        )min(1 dnss

                                                        SVD Compact Representation

                                                        Useful Labeling

                                                        Singular Values in Increasing Order

                                                        Note singular values = 0 can be omitted

                                                        (Since do ldquo0-Stretchingrdquo)

                                                        Review of Linear Algebra (Cont)

                                                        )min(1 dnss

                                                        Review of Linear Algebra (Cont)

                                                        r

                                                        SVD Compact Representation

                                                        Useful Labeling

                                                        Singular Values in Increasing Order

                                                        Note singular values = 0 can be omitted

                                                        Let = of positive singular values

                                                        )min(1 dnss

                                                        Review of Linear Algebra (Cont)

                                                        r

                                                        SVD Compact Representation

                                                        Useful Labeling

                                                        Singular Values in Increasing Order

                                                        Note singular values = 0 can be omitted

                                                        Let = of positive singular values

                                                        Then

                                                        Where are truncations of

                                                        trnrrrd VSUX

                                                        VSU

                                                        )min(1 dnss

                                                        Review of Linear Algebra (Cont)

                                                        SVD Full Representation

                                                        =

                                                        Graphics Display Assumes

                                                        ndX ddU ndS nn

                                                        tV

                                                        nd

                                                        Review of Linear Algebra (Cont)

                                                        SVD Full Representation

                                                        =

                                                        Full Rank Basis Matrix

                                                        ndX ddU ndS nn

                                                        tV

                                                        Review of Linear Algebra (Cont)

                                                        SVD Full Representation

                                                        =

                                                        Full Rank Basis Matrix

                                                        All 0s in Bottom

                                                        ndX ddU ndS nn

                                                        tV

                                                        Review of Linear Algebra (Cont)

                                                        SVD Reduced Representation

                                                        =

                                                        These Columns Get 0ed Out

                                                        ndX ddU nnS nn

                                                        tV

                                                        nnd 0

                                                        Review of Linear Algebra (Cont)

                                                        SVD Reduced Representation

                                                        =ndX ndU nnS nn

                                                        tV

                                                        Review of Linear Algebra (Cont)

                                                        SVD Reduced Representation

                                                        =

                                                        Also Some of These May be 0

                                                        ndX ndU nnS nn

                                                        tV

                                                        Review of Linear Algebra (Cont)

                                                        SVD Compact Representation

                                                        = ndX rdU

                                                        rrS nrtV

                                                        0

                                                        Review of Linear Algebra (Cont)

                                                        SVD Compact Representation

                                                        =

                                                        These Get 0ed Out

                                                        ndX rdU

                                                        rrS nrtV

                                                        0

                                                        Review of Linear Algebra (Cont)

                                                        SVD Compact Representation

                                                        = ndX rdU

                                                        rrS nrtV

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition

                                                        For a (Symmetric) Square Matrix

                                                        Find

                                                        ddX

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition

                                                        For a (Symmetric) Square Matrix

                                                        Find a Diagonal Matrix

                                                        Called Eigenvalues

                                                        Convenient Ordering

                                                        ddX

                                                        d

                                                        D

                                                        0

                                                        01

                                                        n 1

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition

                                                        For a (Symmetric) Square Matrix

                                                        Find a Diagonal Matrix

                                                        And an Orthonormal Matrix

                                                        (ie )

                                                        ddX

                                                        d

                                                        D

                                                        0

                                                        01

                                                        ddB

                                                        ddtt IBBBB

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition

                                                        For a (Symmetric) Square Matrix

                                                        Find a Diagonal Matrix

                                                        And an Orthonormal Matrix

                                                        (ie )

                                                        So that ie

                                                        ddX

                                                        d

                                                        D

                                                        0

                                                        01

                                                        ddB

                                                        ddtt IBBBB

                                                        DBBX tBDBX

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                        (looks similar)

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                        Complex (uses )

                                                        VU

                                                        1i

                                                        Review of Linear Algebra (Cont)

                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                        VU

                                                        X

                                                        BVU

                                                        1i

                                                        • Participant Presentations
                                                        • PCA to find clusters
                                                        • Statistical Smoothing
                                                        • Kernel Density Estimation
                                                        • Scatterplot Smoothing
                                                        • Statistical Smoothing (2)
                                                        • SiZer Background
                                                        • SiZer Background (2)
                                                        • SiZer Background (3)
                                                        • SiZer Background (4)
                                                        • SiZer Background (5)
                                                        • SiZer Overview
                                                        • PCA to find clusters (2)
                                                        • PCA to find clusters (3)
                                                        • PCA to find clusters (4)
                                                        • PCA to find clusters (5)
                                                        • PCA to find clusters (6)
                                                        • PCA to find clusters (7)
                                                        • Recall Yeast Cell Cycle Data
                                                        • Yeast Cell Cycle Data FDA View
                                                        • Yeast Cell Cycle Data FDA View (2)
                                                        • Yeast Cell Cycle Data FDA View (3)
                                                        • Yeast Cell Cycles Freq 2 Proj
                                                        • Frequency 2 Analysis
                                                        • Frequency 2 Analysis (2)
                                                        • Frequency 2 Analysis (3)
                                                        • Yeast Cell Cycle
                                                        • SiZer Study of Distrsquon of Angles
                                                        • Reclassification of Major Genes
                                                        • Compare to Previous Classifrsquon
                                                        • New Subpopulation View
                                                        • New Subpopulation View (2)
                                                        • Detailed Look at PCA
                                                        • PCA Rediscovery ndash Renaming
                                                        • PCA Rediscovery ndash Renaming (2)
                                                        • PCA Rediscovery ndash Renaming (3)
                                                        • PCA Rediscovery ndash Renaming (4)
                                                        • PCA Rediscovery ndash Renaming (5)
                                                        • An Interesting Historical Note
                                                        • An Interesting Historical Note (2)
                                                        • Detailed Look at PCA (2)
                                                        • Detailed Look at PCA (3)
                                                        • Review of Linear Algebra
                                                        • Review of Linear Algebra (2)
                                                        • Review of Linear Algebra (3)
                                                        • Review of Linear Algebra (Cont)
                                                        • Review of Linear Algebra (Cont) (2)
                                                        • Review of Linear Algebra (Cont) (3)
                                                        • Review of Linear Algebra (Cont) (4)
                                                        • Review of Linear Algebra (Cont) (5)
                                                        • Review of Linear Algebra (Cont) (6)
                                                        • Review of Linear Algebra (Cont) (7)
                                                        • Review of Linear Algebra (Cont) (8)
                                                        • Review of Linear Algebra (Cont) (9)
                                                        • Review of Linear Algebra (Cont) (10)
                                                        • Review of Linear Algebra (Cont) (11)
                                                        • Review of Linear Algebra (Cont) (12)
                                                        • Review of Linear Algebra (Cont) (13)
                                                        • Review of Linear Algebra (Cont) (14)
                                                        • Review of Linear Algebra (Cont) (15)
                                                        • Review of Linear Algebra (Cont) (16)
                                                        • Review of Linear Algebra (Cont) (17)
                                                        • Review of Linear Algebra (Cont) (18)
                                                        • Review of Linear Algebra (Cont) (19)
                                                        • Review of Linear Algebra (Cont) (20)
                                                        • Review of Linear Algebra (Cont) (21)
                                                        • Review of Linear Algebra (Cont) (22)
                                                        • Review of Linear Algebra (Cont) (23)
                                                        • Review of Linear Algebra (Cont) (24)
                                                        • Review of Linear Algebra (Cont) (25)
                                                        • Review of Linear Algebra (Cont) (26)
                                                        • Review of Linear Algebra (Cont) (27)
                                                        • Review of Linear Algebra (Cont) (28)
                                                        • Review of Linear Algebra (Cont) (29)
                                                        • Review of Linear Algebra (Cont) (30)
                                                        • Review of Linear Algebra (Cont) (31)
                                                        • Review of Linear Algebra (Cont) (32)
                                                        • Review of Linear Algebra (Cont) (33)
                                                        • Review of Linear Algebra (Cont) (34)
                                                        • Review of Linear Algebra (Cont) (35)
                                                        • Review of Linear Algebra (Cont) (36)
                                                        • Review of Linear Algebra (Cont) (37)
                                                        • Review of Linear Algebra (Cont) (38)
                                                        • Review of Linear Algebra (Cont) (39)
                                                        • Review of Linear Algebra (Cont) (40)
                                                        • Review of Linear Algebra (Cont) (41)
                                                        • Review of Linear Algebra (Cont) (42)
                                                        • Review of Linear Algebra (Cont) (43)
                                                        • Review of Linear Algebra (Cont) (44)
                                                        • Review of Linear Algebra (Cont) (45)
                                                        • Review of Linear Algebra (Cont) (46)
                                                        • Review of Linear Algebra (Cont) (47)
                                                        • Review of Linear Algebra (Cont) (48)
                                                        • Review of Linear Algebra (Cont) (49)
                                                        • Review of Linear Algebra (Cont) (50)
                                                        • Review of Linear Algebra (Cont) (51)
                                                        • Review of Linear Algebra (Cont) (52)
                                                        • Review of Linear Algebra (Cont) (53)
                                                        • Review of Linear Algebra (Cont) (54)
                                                        • Review of Linear Algebra (Cont) (55)
                                                        • Review of Linear Algebra (Cont) (56)
                                                        • Review of Linear Algebra (Cont) (57)
                                                        • Review of Linear Algebra (Cont) (58)
                                                        • Review of Linear Algebra (Cont) (59)
                                                        • Review of Linear Algebra (Cont) (60)
                                                        • Review of Linear Algebra (Cont) (61)
                                                        • Review of Linear Algebra (Cont) (62)
                                                        • Review of Linear Algebra (Cont) (63)
                                                        • Review of Linear Algebra (Cont) (64)
                                                        • Review of Linear Algebra (Cont) (65)
                                                        • Review of Linear Algebra (Cont) (66)
                                                        • Review of Linear Algebra (Cont) (67)
                                                        • Review of Linear Algebra (Cont) (68)
                                                        • Review of Linear Algebra (Cont) (69)
                                                        • Review of Linear Algebra (Cont) (70)
                                                        • Review of Linear Algebra (Cont) (71)
                                                        • Review of Linear Algebra (Cont) (72)
                                                        • Review of Linear Algebra (Cont) (73)
                                                        • Review of Linear Algebra (Cont) (74)
                                                        • Review of Linear Algebra (Cont) (75)
                                                        • Review of Linear Algebra (Cont) (76)
                                                        • Review of Linear Algebra (Cont) (77)
                                                        • Review of Linear Algebra (Cont) (78)
                                                        • Review of Linear Algebra (Cont) (79)
                                                        • Review of Linear Algebra (Cont) (80)
                                                        • Review of Linear Algebra (Cont) (81)

                                                          Reclassification of Major Genes

                                                          Compare to Previous Classifrsquon

                                                          New Subpopulation View

                                                          New Subpopulation View

                                                          NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                          Detailed Look at PCA

                                                          Now Study ldquoFolklorerdquo More Carefully

                                                          bull BackGround

                                                          bull History

                                                          bull Underpinnings

                                                          (Mathematical amp Computational)

                                                          Good Overall Reference Jolliffe (2002)

                                                          PCA Rediscovery ndash Renaming

                                                          Statistics Principal Component Analysis (PCA)

                                                          PCA Rediscovery ndash Renaming

                                                          Statistics Principal Component Analysis (PCA)

                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                          PCA Rediscovery ndash Renaming

                                                          Statistics Principal Component Analysis (PCA)

                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                                          PCA Rediscovery ndash Renaming

                                                          Statistics Principal Component Analysis (PCA)

                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                                          PCA Rediscovery ndash Renaming

                                                          Statistics Principal Component Analysis (PCA)

                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                                          Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                          An Interesting Historical Note

                                                          The 1st () application of PCA to Functional

                                                          Data Analysis

                                                          An Interesting Historical Note

                                                          The 1st () application of PCA to Functional

                                                          Data Analysis

                                                          Rao (1958)

                                                          1st Paper with ldquoCurves as Data Objectsrdquo

                                                          viewpoint

                                                          Detailed Look at PCA

                                                          Three Important (amp Interesting) Viewpoints

                                                          1 Mathematics

                                                          2 Numerics

                                                          3 Statistics

                                                          Goal Study Interrelationships

                                                          Detailed Look at PCA

                                                          Three Important (amp Interesting) Viewpoints

                                                          1 Mathematics

                                                          2 Numerics

                                                          3 Statistics

                                                          1st Review Linear Alg and Multivar Prob

                                                          Review of Linear Algebra

                                                          Vector Space

                                                          bull set of ldquovectorsrdquo

                                                          bull and ldquoscalarsrdquo (coefficients)

                                                          xa

                                                          Review of Linear Algebra

                                                          Vector Space

                                                          bull set of ldquovectorsrdquo

                                                          bull and ldquoscalarsrdquo (coefficients)

                                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                          ( in space)

                                                          xa

                                                          i

                                                          ii xa

                                                          Review of Linear Algebra

                                                          Vector Space

                                                          bull set of ldquovectorsrdquo

                                                          bull and ldquoscalarsrdquo (coefficients)

                                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                          ( in space)

                                                          eg

                                                          ldquo dim Euclidrsquon spacerdquo

                                                          xa

                                                          i

                                                          ii xa

                                                          d

                                                          d

                                                          d xx

                                                          x

                                                          x

                                                          x 1

                                                          1

                                                          d

                                                          Review of Linear Algebra (Cont)

                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                          Review of Linear Algebra (Cont)

                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                          Review of Linear Algebra (Cont)

                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                          Note Planes not Through the Origin

                                                          are not Subspaces

                                                          (Do not Contain )00 x

                                                          Review of Linear Algebra (Cont)

                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                          (all Linear Combos of them =

                                                          = Containing Hyperplane

                                                          through Origin)

                                                          Review of Linear Algebra (Cont)

                                                          Basis of Subspace Set of Vectors that

                                                          bull Span ie Everything is a Lin Com of them

                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                          Review of Linear Algebra (Cont)

                                                          Basis of Subspace Set of Vectors that

                                                          bull Span ie Everything is a Lin Com of them

                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                          bull eg ldquoUnit Vector Basisrdquo

                                                          bull Since

                                                          d

                                                          1

                                                          0

                                                          0

                                                          0

                                                          1

                                                          0

                                                          0

                                                          0

                                                          1

                                                          1

                                                          0

                                                          0

                                                          0

                                                          1

                                                          0

                                                          0

                                                          0

                                                          1

                                                          212

                                                          1

                                                          d

                                                          d

                                                          xxx

                                                          x

                                                          x

                                                          x

                                                          Review of Linear Algebra (Cont)

                                                          Basis Matrix of subspace of

                                                          Given a basis

                                                          create matrix of columns

                                                          dnvv 1

                                                          nddnd

                                                          n

                                                          n

                                                          vv

                                                          vv

                                                          vvB

                                                          1

                                                          111

                                                          1

                                                          Review of Linear Algebra (Cont)

                                                          Then linear combo is a matrix multiplicatrsquon

                                                          where

                                                          n

                                                          iii aBva

                                                          1

                                                          na

                                                          a

                                                          a 1

                                                          Review of Linear Algebra (Cont)

                                                          Then linear combo is a matrix multiplicatrsquon

                                                          where

                                                          Note Right Multiplication Gives

                                                          Linear Combination of Column Vectors

                                                          n

                                                          iii aBva

                                                          1

                                                          na

                                                          a

                                                          a 1

                                                          Review of Linear Algebra (Cont)

                                                          Then linear combo is a matrix multiplicatrsquon

                                                          where

                                                          Check sizes

                                                          n

                                                          iii aBva

                                                          1

                                                          na

                                                          a

                                                          a 1

                                                          )1()(1 nndd

                                                          Review of Linear Algebra (Cont)

                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                          For matrices

                                                          Define the Matrix Product

                                                          mkk

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          nmm

                                                          n

                                                          bb

                                                          bb

                                                          B

                                                          1

                                                          111

                                                          m

                                                          iniik

                                                          m

                                                          iiik

                                                          m

                                                          inii

                                                          m

                                                          iii

                                                          baba

                                                          baba

                                                          AB

                                                          1

                                                          11

                                                          11

                                                          111

                                                          Review of Linear Algebra (Cont)

                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                          For matrices

                                                          Define the Matrix Product

                                                          (Inner Products of Rows With Columns )

                                                          mkk

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          nmm

                                                          n

                                                          bb

                                                          bb

                                                          B

                                                          1

                                                          111

                                                          m

                                                          iniik

                                                          m

                                                          iiik

                                                          m

                                                          inii

                                                          m

                                                          iii

                                                          baba

                                                          baba

                                                          AB

                                                          1

                                                          11

                                                          11

                                                          111

                                                          A B

                                                          Review of Linear Algebra (Cont)

                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                          For matrices

                                                          Define the Matrix Product

                                                          (Inner Products of Rows With Columns )

                                                          (Composition of Linear Transformations)

                                                          mkk

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          nmm

                                                          n

                                                          bb

                                                          bb

                                                          B

                                                          1

                                                          111

                                                          m

                                                          iniik

                                                          m

                                                          iiik

                                                          m

                                                          inii

                                                          m

                                                          iii

                                                          baba

                                                          baba

                                                          AB

                                                          1

                                                          11

                                                          11

                                                          111

                                                          A B

                                                          Review of Linear Algebra (Cont)

                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                          For matrices

                                                          Define the Matrix Product

                                                          (Inner Products of Rows With Columns )

                                                          (Composition of Linear Transformations)

                                                          Often Useful to Check Sizes

                                                          mkk

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          nmm

                                                          n

                                                          bb

                                                          bb

                                                          B

                                                          1

                                                          111

                                                          m

                                                          iniik

                                                          m

                                                          iiik

                                                          m

                                                          inii

                                                          m

                                                          iii

                                                          baba

                                                          baba

                                                          AB

                                                          1

                                                          11

                                                          11

                                                          111

                                                          nmmknk

                                                          A B

                                                          Review of Linear Algebra (Cont)

                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                          For matrices

                                                          Define the Matrix Product

                                                          (Inner Products of Rows With Columns )

                                                          (Composition of Linear Transformations)

                                                          Often Useful to Check Sizes

                                                          mkk

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          nmm

                                                          n

                                                          bb

                                                          bb

                                                          B

                                                          1

                                                          111

                                                          m

                                                          iniik

                                                          m

                                                          iiik

                                                          m

                                                          inii

                                                          m

                                                          iii

                                                          baba

                                                          baba

                                                          AB

                                                          1

                                                          11

                                                          11

                                                          111

                                                          nmmknk

                                                          A B

                                                          Review of Linear Algebra (Cont)

                                                          Matrix Trace

                                                          bull For a Square Matrix

                                                          bull Define

                                                          mmm

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          m

                                                          iiiaAtr

                                                          1)(

                                                          Review of Linear Algebra (Cont)

                                                          Matrix Trace

                                                          bull For a Square Matrix

                                                          bull Define

                                                          bull Trace Commutes with Matrix Multiplication

                                                          mmm

                                                          m

                                                          aa

                                                          aa

                                                          A

                                                          1

                                                          111

                                                          m

                                                          iiiaAtr

                                                          1)(

                                                          BAtrABtr

                                                          Review of Linear Algebra (Cont)

                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                          bull Number of Elements in a Basis (Unique)

                                                          Review of Linear Algebra (Cont)

                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                          bull Number of Elements in a Basis (Unique)

                                                          bull (Use Basis Above)

                                                          bull eg dim of a line is 1

                                                          bull eg dim of a plane is 2

                                                          dd dim

                                                          Review of Linear Algebra (Cont)

                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                          bull Number of Elements in a Basis (Unique)

                                                          bull (Use Basis Above)

                                                          bull eg dim of a line is 1

                                                          bull eg dim of a plane is 2

                                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                                          (in Statistical Uses eg ANOVA)

                                                          dd dim

                                                          Review of Linear Algebra (Cont)

                                                          Norm of a Vector

                                                          bull in d 21

                                                          21

                                                          1

                                                          2 xxxx td

                                                          jj

                                                          Review of Linear Algebra (Cont)

                                                          Norm of a Vector

                                                          bull in

                                                          bull Idea length of the vector

                                                          d 2121

                                                          1

                                                          2 xxxx td

                                                          jj

                                                          Review of Linear Algebra (Cont)

                                                          Norm of a Vector

                                                          bull in

                                                          bull Idea length of the vector

                                                          bull Note strange properties for high

                                                          eg ldquolength of diagonal of unit cuberdquo =

                                                          d 2121

                                                          1

                                                          2 xxxx td

                                                          jj

                                                          d

                                                          d

                                                          Review of Linear Algebra (Cont)

                                                          Norm of a Vector (cont)

                                                          bull Length Normalized Vector

                                                          (has Length 1 thus on Surf of Unit Sphere

                                                          amp is a Direction Vector)

                                                          x

                                                          x

                                                          Review of Linear Algebra (Cont)

                                                          Norm of a Vector (cont)

                                                          bull Length Normalized Vector

                                                          (has Length 1 thus on Surf of Unit Sphere

                                                          amp is a Direction Vector)

                                                          bull Define Distance as

                                                          x

                                                          x

                                                          yxyxyxyxd t

                                                          Review of Linear Algebra (Cont)

                                                          Inner (Dot Scalar) Product

                                                          bull for Vectors and

                                                          yxyxyx td

                                                          jjj

                                                          1

                                                          x y

                                                          Review of Linear Algebra (Cont)

                                                          Inner (Dot Scalar) Product

                                                          bull for Vectors and

                                                          bull Related to Norm via

                                                          yxyxyx td

                                                          jjj

                                                          1

                                                          x y

                                                          21 xxx

                                                          Review of Linear Algebra (Cont)

                                                          Inner (Dot Scalar) Product (cont)

                                                          bull measures ldquoangle between and rdquo as

                                                          yyxx

                                                          yx

                                                          yx

                                                          yxyxangle

                                                          tt

                                                          t

                                                          11 cos

                                                          cos

                                                          x y

                                                          Review of Linear Algebra (Cont)

                                                          Inner (Dot Scalar) Product (cont)

                                                          bull measures ldquoangle between and rdquo as

                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                          if and only if

                                                          yyxx

                                                          yx

                                                          yx

                                                          yxyxangle

                                                          tt

                                                          t

                                                          11 cos

                                                          cos

                                                          x y

                                                          yx 0 yx

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis

                                                          bull All Orthogonal to each other

                                                          ie for

                                                          bull All have Length 1

                                                          ie for

                                                          nvv 1

                                                          1 ii vv

                                                          0 ii vv ii

                                                          ni 1

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis (cont)

                                                          bull Spectral Representation

                                                          where

                                                          nvv 1

                                                          n

                                                          iii vax

                                                          1

                                                          ii vxa

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis (cont)

                                                          bull Spectral Representation

                                                          where

                                                          (Coefficient is Inner Product Cool Notation)

                                                          nvv 1

                                                          n

                                                          iii vax

                                                          1

                                                          ii vxa

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis (cont)

                                                          bull Spectral Representation

                                                          where

                                                          Check

                                                          nvv 1

                                                          n

                                                          iii vax

                                                          1

                                                          ii vxa

                                                          iii

                                                          n

                                                          iii

                                                          n

                                                          iiii avvavvavx

                                                          1

                                                          1

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis (cont)

                                                          bull Spectral Representation

                                                          where

                                                          Check

                                                          bull Matrix Notation where ie

                                                          For the Basis Matrix

                                                          nvv 1

                                                          n

                                                          iii vax

                                                          1

                                                          ii vxa

                                                          iii

                                                          n

                                                          iii

                                                          n

                                                          iiii avvavvavx

                                                          1

                                                          1

                                                          aBx Bxa tt xBa t

                                                          nvvB 1

                                                          Review of Linear Algebra (Cont)

                                                          Orthonormal Basis (cont)

                                                          bull Spectral Representation

                                                          where

                                                          Check

                                                          bull Matrix Notation where ie

                                                          is called transform of

                                                          (eg Fourier or Wavelet)

                                                          nvv 1

                                                          n

                                                          iii vax

                                                          1

                                                          ii vxa

                                                          iii

                                                          n

                                                          iii

                                                          n

                                                          iiii avvavvavx

                                                          1

                                                          1

                                                          aBx Bxa tt xBa t

                                                          xa

                                                          Review of Linear Algebra (Cont)

                                                          Parseval identity for

                                                          in subsp genrsquod by o n basis

                                                          x

                                                          nvv 1

                                                          2

                                                          1

                                                          22

                                                          1

                                                          2 aavxx

                                                          n

                                                          ii

                                                          n

                                                          ii

                                                          Review of Linear Algebra (Cont)

                                                          Parseval identity for

                                                          in subsp genrsquod by o n basis

                                                          bull Pythagorean theorem

                                                          bull ldquoDecomposition of Energyrdquo

                                                          bull ANOVA - sums of squares

                                                          x

                                                          nvv 1

                                                          2

                                                          1

                                                          22

                                                          1

                                                          2 aavxx

                                                          n

                                                          ii

                                                          n

                                                          ii

                                                          Review of Linear Algebra (Cont)

                                                          Parseval identity for

                                                          in subsp genrsquod by o n basis

                                                          bull Pythagorean theorem

                                                          bull ldquoDecomposition of Energyrdquo

                                                          bull ANOVA - sums of squares

                                                          bull Transform has same length as

                                                          ie ldquorotation in rdquo

                                                          x

                                                          nvv 1

                                                          2

                                                          1

                                                          22

                                                          1

                                                          2 aavxx

                                                          n

                                                          ii

                                                          n

                                                          ii

                                                          a xd

                                                          Projection of a Vector onto a Subspace

                                                          bull Idea Member of that is Closest to

                                                          (ie ldquoBest Approxrsquonrdquo)

                                                          Review of Linear Algebra (Cont)x

                                                          xV

                                                          V

                                                          Projection of a Vector onto a Subspace

                                                          bull Idea Member of that is Closest to

                                                          (ie ldquoBest Approxrsquonrdquo)

                                                          bull Find that Solves

                                                          (ldquoLeast Squaresrdquo)

                                                          Review of Linear Algebra (Cont)x

                                                          xV

                                                          V

                                                          VxPV vxVv

                                                          min

                                                          Projection of a Vector onto a Subspace

                                                          bull Idea Member of that is Closest to

                                                          (ie ldquoBest Approxrsquonrdquo)

                                                          bull Find that Solves

                                                          (ldquoLeast Squaresrdquo)

                                                          bull For Inner Product (Hilbert) Space

                                                          Exists and is Unique

                                                          Review of Linear Algebra (Cont)x

                                                          xV

                                                          V

                                                          VxPV vxVv

                                                          min

                                                          xPV

                                                          Projection of a Vector onto a Subspace (cont)

                                                          bull General Solution in for Basis Matrix

                                                          Review of Linear Algebra (Cont)

                                                          d VB

                                                          xBBBBxP tVV

                                                          tVVV

                                                          1

                                                          Projection of a Vector onto a Subspace (cont)

                                                          bull General Solution in for Basis Matrix

                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                          (thus projection is another linear operation)

                                                          Review of Linear Algebra (Cont)

                                                          d VB

                                                          xBBBBxP tVV

                                                          tVVV

                                                          1

                                                          tVV

                                                          tVVV BBBBP

                                                          1

                                                          Projection of a Vector onto a Subspace (cont)

                                                          bull General Solution in for Basis Matrix

                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                          (thus projection is another linear operation)

                                                          (note same operation underlies least squares)

                                                          Review of Linear Algebra (Cont)

                                                          d VB

                                                          xBBBBxP tVV

                                                          tVVV

                                                          1

                                                          tVV

                                                          tVVV BBBBP

                                                          1

                                                          Review of Linear Algebra (Cont)

                                                          Projection using Orthonormal Basis

                                                          bull Basis Matrix is Orthonormal

                                                          nnVtV IBB

                                                          10

                                                          01

                                                          1

                                                          111

                                                          1

                                                          1

                                                          nnn

                                                          n

                                                          ntn

                                                          t

                                                          vvvv

                                                          vvvv

                                                          vv

                                                          v

                                                          v

                                                          nvv 1

                                                          Review of Linear Algebra (Cont)

                                                          Projection using Orthonormal Basis

                                                          bull Basis Matrix is Orthonormal

                                                          bull So =

                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                          (Recall Right Multrsquon)

                                                          nnVtV IBB

                                                          10

                                                          01

                                                          1

                                                          111

                                                          1

                                                          1

                                                          nnn

                                                          n

                                                          ntn

                                                          t

                                                          vvvv

                                                          vvvv

                                                          vv

                                                          v

                                                          v

                                                          xBBxP tVVV

                                                          x V

                                                          nvv 1

                                                          Review of Linear Algebra (Cont)

                                                          Projection using Orthonormal Basis (cont)

                                                          bull For Orthogonal Complement

                                                          and

                                                          V

                                                          xPxPx VV 222xPxPx VV

                                                          Review of Linear Algebra (Cont)

                                                          Projection using Orthonormal Basis (cont)

                                                          bull For Orthogonal Complement

                                                          and

                                                          bull Parseval Inequality

                                                          V

                                                          xPxPx VV 222xPxPx VV

                                                          2

                                                          1

                                                          22

                                                          1

                                                          22 aavxxxP

                                                          n

                                                          ii

                                                          n

                                                          iiV

                                                          Review of Linear Algebra (Cont)

                                                          (Real) Unitary Matrices with

                                                          bull Orthonormal Basis Matrix

                                                          (So All of Above Applies)

                                                          ddU IUU t

                                                          Review of Linear Algebra (Cont)

                                                          (Real) Unitary Matrices with

                                                          bull Orthonormal Basis Matrix

                                                          (So All of Above Applies)

                                                          bull Note Transformrsquon is Distance Preserving

                                                          ddU IUU t

                                                          yxdyxyxyxUyUxUdn

                                                          i ii 2

                                                          1

                                                          Review of Linear Algebra (Cont)

                                                          (Real) Unitary Matrices with

                                                          bull Orthonormal Basis Matrix

                                                          (So All of Above Applies)

                                                          bull Note Transformrsquon is Distance Preserving

                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                          ddU IUU t

                                                          yxdyxyxyxUyUxUdn

                                                          i ii 2

                                                          1

                                                          U

                                                          Review of Linear Algebra (Cont)

                                                          Singular Value Decomposition (SVD)

                                                          For a Matrix

                                                          Find

                                                          ndX

                                                          Review of Linear Algebra (Cont)

                                                          Singular Value Decomposition (SVD)

                                                          For a Matrix

                                                          Find a Diagonal Matrix

                                                          with Entries

                                                          called Singular Values

                                                          ndX

                                                          ndS

                                                          )min(1 ndss

                                                          Review of Linear Algebra (Cont)

                                                          Singular Value Decomposition (SVD)

                                                          For a Matrix

                                                          Find a Diagonal Matrix

                                                          with Entries

                                                          called Singular Values

                                                          And Unitary (Rotation) Matrices

                                                          (recall )

                                                          ndX

                                                          ndS

                                                          )min(1 ndss

                                                          ddU nnV

                                                          IVVUU tt

                                                          Review of Linear Algebra (Cont)

                                                          Singular Value Decomposition (SVD)

                                                          For a Matrix

                                                          Find a Diagonal Matrix

                                                          with Entries

                                                          called Singular Values

                                                          And Unitary (Rotation) Matrices

                                                          (recall )

                                                          So That

                                                          ndX

                                                          ndS

                                                          )min(1 ndss

                                                          ddU nnV

                                                          IVVUU tt tUSVX

                                                          Review of Linear Algebra (Cont)

                                                          Intuition behind Singular Value Decomposition

                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                          vVSUvVSUvX tt

                                                          Review of Linear Algebra (Cont)

                                                          Intuition behind Singular Value Decomposition

                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                          bull First rotate

                                                          X

                                                          vVSUvVSUvX tt

                                                          Review of Linear Algebra (Cont)

                                                          Intuition behind Singular Value Decomposition

                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                          bull First rotate

                                                          bull Second rescale coordinate axes (by )

                                                          X

                                                          vVSUvVSUvX tt

                                                          is

                                                          Review of Linear Algebra (Cont)

                                                          Intuition behind Singular Value Decomposition

                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                          bull First rotate

                                                          bull Second rescale coordinate axes (by )

                                                          bull Third rotate again

                                                          X

                                                          vVSUvVSUvX tt

                                                          is

                                                          Review of Linear Algebra (Cont)

                                                          Intuition behind Singular Value Decomposition

                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                          bull First rotate

                                                          bull Second rescale coordinate axes (by )

                                                          bull Third rotate again

                                                          bull ie have diagonalized the transformation

                                                          X

                                                          vVSUvVSUvX tt

                                                          is

                                                          SVD Compact Representation

                                                          Useful Labeling

                                                          Singular Values in Increasing Order

                                                          Review of Linear Algebra (Cont)

                                                          )min(1 dnss

                                                          SVD Compact Representation

                                                          Useful Labeling

                                                          Singular Values in Increasing Order

                                                          Note singular values = 0 can be omitted

                                                          (Since do ldquo0-Stretchingrdquo)

                                                          Review of Linear Algebra (Cont)

                                                          )min(1 dnss

                                                          Review of Linear Algebra (Cont)

                                                          r

                                                          SVD Compact Representation

                                                          Useful Labeling

                                                          Singular Values in Increasing Order

                                                          Note singular values = 0 can be omitted

                                                          Let = of positive singular values

                                                          )min(1 dnss

                                                          Review of Linear Algebra (Cont)

                                                          r

                                                          SVD Compact Representation

                                                          Useful Labeling

                                                          Singular Values in Increasing Order

                                                          Note singular values = 0 can be omitted

                                                          Let = of positive singular values

                                                          Then

                                                          Where are truncations of

                                                          trnrrrd VSUX

                                                          VSU

                                                          )min(1 dnss

                                                          Review of Linear Algebra (Cont)

                                                          SVD Full Representation

                                                          =

                                                          Graphics Display Assumes

                                                          ndX ddU ndS nn

                                                          tV

                                                          nd

                                                          Review of Linear Algebra (Cont)

                                                          SVD Full Representation

                                                          =

                                                          Full Rank Basis Matrix

                                                          ndX ddU ndS nn

                                                          tV

                                                          Review of Linear Algebra (Cont)

                                                          SVD Full Representation

                                                          =

                                                          Full Rank Basis Matrix

                                                          All 0s in Bottom

                                                          ndX ddU ndS nn

                                                          tV

                                                          Review of Linear Algebra (Cont)

                                                          SVD Reduced Representation

                                                          =

                                                          These Columns Get 0ed Out

                                                          ndX ddU nnS nn

                                                          tV

                                                          nnd 0

                                                          Review of Linear Algebra (Cont)

                                                          SVD Reduced Representation

                                                          =ndX ndU nnS nn

                                                          tV

                                                          Review of Linear Algebra (Cont)

                                                          SVD Reduced Representation

                                                          =

                                                          Also Some of These May be 0

                                                          ndX ndU nnS nn

                                                          tV

                                                          Review of Linear Algebra (Cont)

                                                          SVD Compact Representation

                                                          = ndX rdU

                                                          rrS nrtV

                                                          0

                                                          Review of Linear Algebra (Cont)

                                                          SVD Compact Representation

                                                          =

                                                          These Get 0ed Out

                                                          ndX rdU

                                                          rrS nrtV

                                                          0

                                                          Review of Linear Algebra (Cont)

                                                          SVD Compact Representation

                                                          = ndX rdU

                                                          rrS nrtV

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition

                                                          For a (Symmetric) Square Matrix

                                                          Find

                                                          ddX

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition

                                                          For a (Symmetric) Square Matrix

                                                          Find a Diagonal Matrix

                                                          Called Eigenvalues

                                                          Convenient Ordering

                                                          ddX

                                                          d

                                                          D

                                                          0

                                                          01

                                                          n 1

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition

                                                          For a (Symmetric) Square Matrix

                                                          Find a Diagonal Matrix

                                                          And an Orthonormal Matrix

                                                          (ie )

                                                          ddX

                                                          d

                                                          D

                                                          0

                                                          01

                                                          ddB

                                                          ddtt IBBBB

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition

                                                          For a (Symmetric) Square Matrix

                                                          Find a Diagonal Matrix

                                                          And an Orthonormal Matrix

                                                          (ie )

                                                          So that ie

                                                          ddX

                                                          d

                                                          D

                                                          0

                                                          01

                                                          ddB

                                                          ddtt IBBBB

                                                          DBBX tBDBX

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                          (looks similar)

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                          Complex (uses )

                                                          VU

                                                          1i

                                                          Review of Linear Algebra (Cont)

                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                          VU

                                                          X

                                                          BVU

                                                          1i

                                                          • Participant Presentations
                                                          • PCA to find clusters
                                                          • Statistical Smoothing
                                                          • Kernel Density Estimation
                                                          • Scatterplot Smoothing
                                                          • Statistical Smoothing (2)
                                                          • SiZer Background
                                                          • SiZer Background (2)
                                                          • SiZer Background (3)
                                                          • SiZer Background (4)
                                                          • SiZer Background (5)
                                                          • SiZer Overview
                                                          • PCA to find clusters (2)
                                                          • PCA to find clusters (3)
                                                          • PCA to find clusters (4)
                                                          • PCA to find clusters (5)
                                                          • PCA to find clusters (6)
                                                          • PCA to find clusters (7)
                                                          • Recall Yeast Cell Cycle Data
                                                          • Yeast Cell Cycle Data FDA View
                                                          • Yeast Cell Cycle Data FDA View (2)
                                                          • Yeast Cell Cycle Data FDA View (3)
                                                          • Yeast Cell Cycles Freq 2 Proj
                                                          • Frequency 2 Analysis
                                                          • Frequency 2 Analysis (2)
                                                          • Frequency 2 Analysis (3)
                                                          • Yeast Cell Cycle
                                                          • SiZer Study of Distrsquon of Angles
                                                          • Reclassification of Major Genes
                                                          • Compare to Previous Classifrsquon
                                                          • New Subpopulation View
                                                          • New Subpopulation View (2)
                                                          • Detailed Look at PCA
                                                          • PCA Rediscovery ndash Renaming
                                                          • PCA Rediscovery ndash Renaming (2)
                                                          • PCA Rediscovery ndash Renaming (3)
                                                          • PCA Rediscovery ndash Renaming (4)
                                                          • PCA Rediscovery ndash Renaming (5)
                                                          • An Interesting Historical Note
                                                          • An Interesting Historical Note (2)
                                                          • Detailed Look at PCA (2)
                                                          • Detailed Look at PCA (3)
                                                          • Review of Linear Algebra
                                                          • Review of Linear Algebra (2)
                                                          • Review of Linear Algebra (3)
                                                          • Review of Linear Algebra (Cont)
                                                          • Review of Linear Algebra (Cont) (2)
                                                          • Review of Linear Algebra (Cont) (3)
                                                          • Review of Linear Algebra (Cont) (4)
                                                          • Review of Linear Algebra (Cont) (5)
                                                          • Review of Linear Algebra (Cont) (6)
                                                          • Review of Linear Algebra (Cont) (7)
                                                          • Review of Linear Algebra (Cont) (8)
                                                          • Review of Linear Algebra (Cont) (9)
                                                          • Review of Linear Algebra (Cont) (10)
                                                          • Review of Linear Algebra (Cont) (11)
                                                          • Review of Linear Algebra (Cont) (12)
                                                          • Review of Linear Algebra (Cont) (13)
                                                          • Review of Linear Algebra (Cont) (14)
                                                          • Review of Linear Algebra (Cont) (15)
                                                          • Review of Linear Algebra (Cont) (16)
                                                          • Review of Linear Algebra (Cont) (17)
                                                          • Review of Linear Algebra (Cont) (18)
                                                          • Review of Linear Algebra (Cont) (19)
                                                          • Review of Linear Algebra (Cont) (20)
                                                          • Review of Linear Algebra (Cont) (21)
                                                          • Review of Linear Algebra (Cont) (22)
                                                          • Review of Linear Algebra (Cont) (23)
                                                          • Review of Linear Algebra (Cont) (24)
                                                          • Review of Linear Algebra (Cont) (25)
                                                          • Review of Linear Algebra (Cont) (26)
                                                          • Review of Linear Algebra (Cont) (27)
                                                          • Review of Linear Algebra (Cont) (28)
                                                          • Review of Linear Algebra (Cont) (29)
                                                          • Review of Linear Algebra (Cont) (30)
                                                          • Review of Linear Algebra (Cont) (31)
                                                          • Review of Linear Algebra (Cont) (32)
                                                          • Review of Linear Algebra (Cont) (33)
                                                          • Review of Linear Algebra (Cont) (34)
                                                          • Review of Linear Algebra (Cont) (35)
                                                          • Review of Linear Algebra (Cont) (36)
                                                          • Review of Linear Algebra (Cont) (37)
                                                          • Review of Linear Algebra (Cont) (38)
                                                          • Review of Linear Algebra (Cont) (39)
                                                          • Review of Linear Algebra (Cont) (40)
                                                          • Review of Linear Algebra (Cont) (41)
                                                          • Review of Linear Algebra (Cont) (42)
                                                          • Review of Linear Algebra (Cont) (43)
                                                          • Review of Linear Algebra (Cont) (44)
                                                          • Review of Linear Algebra (Cont) (45)
                                                          • Review of Linear Algebra (Cont) (46)
                                                          • Review of Linear Algebra (Cont) (47)
                                                          • Review of Linear Algebra (Cont) (48)
                                                          • Review of Linear Algebra (Cont) (49)
                                                          • Review of Linear Algebra (Cont) (50)
                                                          • Review of Linear Algebra (Cont) (51)
                                                          • Review of Linear Algebra (Cont) (52)
                                                          • Review of Linear Algebra (Cont) (53)
                                                          • Review of Linear Algebra (Cont) (54)
                                                          • Review of Linear Algebra (Cont) (55)
                                                          • Review of Linear Algebra (Cont) (56)
                                                          • Review of Linear Algebra (Cont) (57)
                                                          • Review of Linear Algebra (Cont) (58)
                                                          • Review of Linear Algebra (Cont) (59)
                                                          • Review of Linear Algebra (Cont) (60)
                                                          • Review of Linear Algebra (Cont) (61)
                                                          • Review of Linear Algebra (Cont) (62)
                                                          • Review of Linear Algebra (Cont) (63)
                                                          • Review of Linear Algebra (Cont) (64)
                                                          • Review of Linear Algebra (Cont) (65)
                                                          • Review of Linear Algebra (Cont) (66)
                                                          • Review of Linear Algebra (Cont) (67)
                                                          • Review of Linear Algebra (Cont) (68)
                                                          • Review of Linear Algebra (Cont) (69)
                                                          • Review of Linear Algebra (Cont) (70)
                                                          • Review of Linear Algebra (Cont) (71)
                                                          • Review of Linear Algebra (Cont) (72)
                                                          • Review of Linear Algebra (Cont) (73)
                                                          • Review of Linear Algebra (Cont) (74)
                                                          • Review of Linear Algebra (Cont) (75)
                                                          • Review of Linear Algebra (Cont) (76)
                                                          • Review of Linear Algebra (Cont) (77)
                                                          • Review of Linear Algebra (Cont) (78)
                                                          • Review of Linear Algebra (Cont) (79)
                                                          • Review of Linear Algebra (Cont) (80)
                                                          • Review of Linear Algebra (Cont) (81)

                                                            Compare to Previous Classifrsquon

                                                            New Subpopulation View

                                                            New Subpopulation View

                                                            NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                            Detailed Look at PCA

                                                            Now Study ldquoFolklorerdquo More Carefully

                                                            bull BackGround

                                                            bull History

                                                            bull Underpinnings

                                                            (Mathematical amp Computational)

                                                            Good Overall Reference Jolliffe (2002)

                                                            PCA Rediscovery ndash Renaming

                                                            Statistics Principal Component Analysis (PCA)

                                                            PCA Rediscovery ndash Renaming

                                                            Statistics Principal Component Analysis (PCA)

                                                            Social Sciences Factor Analysis (PCA is a subset)

                                                            PCA Rediscovery ndash Renaming

                                                            Statistics Principal Component Analysis (PCA)

                                                            Social Sciences Factor Analysis (PCA is a subset)

                                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                                            PCA Rediscovery ndash Renaming

                                                            Statistics Principal Component Analysis (PCA)

                                                            Social Sciences Factor Analysis (PCA is a subset)

                                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                                            Applied MathematicsProper Orthogonal Decomposition (POD)

                                                            PCA Rediscovery ndash Renaming

                                                            Statistics Principal Component Analysis (PCA)

                                                            Social Sciences Factor Analysis (PCA is a subset)

                                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                                            Applied MathematicsProper Orthogonal Decomposition (POD)

                                                            Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                            An Interesting Historical Note

                                                            The 1st () application of PCA to Functional

                                                            Data Analysis

                                                            An Interesting Historical Note

                                                            The 1st () application of PCA to Functional

                                                            Data Analysis

                                                            Rao (1958)

                                                            1st Paper with ldquoCurves as Data Objectsrdquo

                                                            viewpoint

                                                            Detailed Look at PCA

                                                            Three Important (amp Interesting) Viewpoints

                                                            1 Mathematics

                                                            2 Numerics

                                                            3 Statistics

                                                            Goal Study Interrelationships

                                                            Detailed Look at PCA

                                                            Three Important (amp Interesting) Viewpoints

                                                            1 Mathematics

                                                            2 Numerics

                                                            3 Statistics

                                                            1st Review Linear Alg and Multivar Prob

                                                            Review of Linear Algebra

                                                            Vector Space

                                                            bull set of ldquovectorsrdquo

                                                            bull and ldquoscalarsrdquo (coefficients)

                                                            xa

                                                            Review of Linear Algebra

                                                            Vector Space

                                                            bull set of ldquovectorsrdquo

                                                            bull and ldquoscalarsrdquo (coefficients)

                                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                            ( in space)

                                                            xa

                                                            i

                                                            ii xa

                                                            Review of Linear Algebra

                                                            Vector Space

                                                            bull set of ldquovectorsrdquo

                                                            bull and ldquoscalarsrdquo (coefficients)

                                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                            ( in space)

                                                            eg

                                                            ldquo dim Euclidrsquon spacerdquo

                                                            xa

                                                            i

                                                            ii xa

                                                            d

                                                            d

                                                            d xx

                                                            x

                                                            x

                                                            x 1

                                                            1

                                                            d

                                                            Review of Linear Algebra (Cont)

                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                            Review of Linear Algebra (Cont)

                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                            Review of Linear Algebra (Cont)

                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                            Note Planes not Through the Origin

                                                            are not Subspaces

                                                            (Do not Contain )00 x

                                                            Review of Linear Algebra (Cont)

                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                            (all Linear Combos of them =

                                                            = Containing Hyperplane

                                                            through Origin)

                                                            Review of Linear Algebra (Cont)

                                                            Basis of Subspace Set of Vectors that

                                                            bull Span ie Everything is a Lin Com of them

                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                            Review of Linear Algebra (Cont)

                                                            Basis of Subspace Set of Vectors that

                                                            bull Span ie Everything is a Lin Com of them

                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                            bull eg ldquoUnit Vector Basisrdquo

                                                            bull Since

                                                            d

                                                            1

                                                            0

                                                            0

                                                            0

                                                            1

                                                            0

                                                            0

                                                            0

                                                            1

                                                            1

                                                            0

                                                            0

                                                            0

                                                            1

                                                            0

                                                            0

                                                            0

                                                            1

                                                            212

                                                            1

                                                            d

                                                            d

                                                            xxx

                                                            x

                                                            x

                                                            x

                                                            Review of Linear Algebra (Cont)

                                                            Basis Matrix of subspace of

                                                            Given a basis

                                                            create matrix of columns

                                                            dnvv 1

                                                            nddnd

                                                            n

                                                            n

                                                            vv

                                                            vv

                                                            vvB

                                                            1

                                                            111

                                                            1

                                                            Review of Linear Algebra (Cont)

                                                            Then linear combo is a matrix multiplicatrsquon

                                                            where

                                                            n

                                                            iii aBva

                                                            1

                                                            na

                                                            a

                                                            a 1

                                                            Review of Linear Algebra (Cont)

                                                            Then linear combo is a matrix multiplicatrsquon

                                                            where

                                                            Note Right Multiplication Gives

                                                            Linear Combination of Column Vectors

                                                            n

                                                            iii aBva

                                                            1

                                                            na

                                                            a

                                                            a 1

                                                            Review of Linear Algebra (Cont)

                                                            Then linear combo is a matrix multiplicatrsquon

                                                            where

                                                            Check sizes

                                                            n

                                                            iii aBva

                                                            1

                                                            na

                                                            a

                                                            a 1

                                                            )1()(1 nndd

                                                            Review of Linear Algebra (Cont)

                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                            For matrices

                                                            Define the Matrix Product

                                                            mkk

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            nmm

                                                            n

                                                            bb

                                                            bb

                                                            B

                                                            1

                                                            111

                                                            m

                                                            iniik

                                                            m

                                                            iiik

                                                            m

                                                            inii

                                                            m

                                                            iii

                                                            baba

                                                            baba

                                                            AB

                                                            1

                                                            11

                                                            11

                                                            111

                                                            Review of Linear Algebra (Cont)

                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                            For matrices

                                                            Define the Matrix Product

                                                            (Inner Products of Rows With Columns )

                                                            mkk

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            nmm

                                                            n

                                                            bb

                                                            bb

                                                            B

                                                            1

                                                            111

                                                            m

                                                            iniik

                                                            m

                                                            iiik

                                                            m

                                                            inii

                                                            m

                                                            iii

                                                            baba

                                                            baba

                                                            AB

                                                            1

                                                            11

                                                            11

                                                            111

                                                            A B

                                                            Review of Linear Algebra (Cont)

                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                            For matrices

                                                            Define the Matrix Product

                                                            (Inner Products of Rows With Columns )

                                                            (Composition of Linear Transformations)

                                                            mkk

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            nmm

                                                            n

                                                            bb

                                                            bb

                                                            B

                                                            1

                                                            111

                                                            m

                                                            iniik

                                                            m

                                                            iiik

                                                            m

                                                            inii

                                                            m

                                                            iii

                                                            baba

                                                            baba

                                                            AB

                                                            1

                                                            11

                                                            11

                                                            111

                                                            A B

                                                            Review of Linear Algebra (Cont)

                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                            For matrices

                                                            Define the Matrix Product

                                                            (Inner Products of Rows With Columns )

                                                            (Composition of Linear Transformations)

                                                            Often Useful to Check Sizes

                                                            mkk

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            nmm

                                                            n

                                                            bb

                                                            bb

                                                            B

                                                            1

                                                            111

                                                            m

                                                            iniik

                                                            m

                                                            iiik

                                                            m

                                                            inii

                                                            m

                                                            iii

                                                            baba

                                                            baba

                                                            AB

                                                            1

                                                            11

                                                            11

                                                            111

                                                            nmmknk

                                                            A B

                                                            Review of Linear Algebra (Cont)

                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                            For matrices

                                                            Define the Matrix Product

                                                            (Inner Products of Rows With Columns )

                                                            (Composition of Linear Transformations)

                                                            Often Useful to Check Sizes

                                                            mkk

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            nmm

                                                            n

                                                            bb

                                                            bb

                                                            B

                                                            1

                                                            111

                                                            m

                                                            iniik

                                                            m

                                                            iiik

                                                            m

                                                            inii

                                                            m

                                                            iii

                                                            baba

                                                            baba

                                                            AB

                                                            1

                                                            11

                                                            11

                                                            111

                                                            nmmknk

                                                            A B

                                                            Review of Linear Algebra (Cont)

                                                            Matrix Trace

                                                            bull For a Square Matrix

                                                            bull Define

                                                            mmm

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            m

                                                            iiiaAtr

                                                            1)(

                                                            Review of Linear Algebra (Cont)

                                                            Matrix Trace

                                                            bull For a Square Matrix

                                                            bull Define

                                                            bull Trace Commutes with Matrix Multiplication

                                                            mmm

                                                            m

                                                            aa

                                                            aa

                                                            A

                                                            1

                                                            111

                                                            m

                                                            iiiaAtr

                                                            1)(

                                                            BAtrABtr

                                                            Review of Linear Algebra (Cont)

                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                            bull Number of Elements in a Basis (Unique)

                                                            Review of Linear Algebra (Cont)

                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                            bull Number of Elements in a Basis (Unique)

                                                            bull (Use Basis Above)

                                                            bull eg dim of a line is 1

                                                            bull eg dim of a plane is 2

                                                            dd dim

                                                            Review of Linear Algebra (Cont)

                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                            bull Number of Elements in a Basis (Unique)

                                                            bull (Use Basis Above)

                                                            bull eg dim of a line is 1

                                                            bull eg dim of a plane is 2

                                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                                            (in Statistical Uses eg ANOVA)

                                                            dd dim

                                                            Review of Linear Algebra (Cont)

                                                            Norm of a Vector

                                                            bull in d 21

                                                            21

                                                            1

                                                            2 xxxx td

                                                            jj

                                                            Review of Linear Algebra (Cont)

                                                            Norm of a Vector

                                                            bull in

                                                            bull Idea length of the vector

                                                            d 2121

                                                            1

                                                            2 xxxx td

                                                            jj

                                                            Review of Linear Algebra (Cont)

                                                            Norm of a Vector

                                                            bull in

                                                            bull Idea length of the vector

                                                            bull Note strange properties for high

                                                            eg ldquolength of diagonal of unit cuberdquo =

                                                            d 2121

                                                            1

                                                            2 xxxx td

                                                            jj

                                                            d

                                                            d

                                                            Review of Linear Algebra (Cont)

                                                            Norm of a Vector (cont)

                                                            bull Length Normalized Vector

                                                            (has Length 1 thus on Surf of Unit Sphere

                                                            amp is a Direction Vector)

                                                            x

                                                            x

                                                            Review of Linear Algebra (Cont)

                                                            Norm of a Vector (cont)

                                                            bull Length Normalized Vector

                                                            (has Length 1 thus on Surf of Unit Sphere

                                                            amp is a Direction Vector)

                                                            bull Define Distance as

                                                            x

                                                            x

                                                            yxyxyxyxd t

                                                            Review of Linear Algebra (Cont)

                                                            Inner (Dot Scalar) Product

                                                            bull for Vectors and

                                                            yxyxyx td

                                                            jjj

                                                            1

                                                            x y

                                                            Review of Linear Algebra (Cont)

                                                            Inner (Dot Scalar) Product

                                                            bull for Vectors and

                                                            bull Related to Norm via

                                                            yxyxyx td

                                                            jjj

                                                            1

                                                            x y

                                                            21 xxx

                                                            Review of Linear Algebra (Cont)

                                                            Inner (Dot Scalar) Product (cont)

                                                            bull measures ldquoangle between and rdquo as

                                                            yyxx

                                                            yx

                                                            yx

                                                            yxyxangle

                                                            tt

                                                            t

                                                            11 cos

                                                            cos

                                                            x y

                                                            Review of Linear Algebra (Cont)

                                                            Inner (Dot Scalar) Product (cont)

                                                            bull measures ldquoangle between and rdquo as

                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                            if and only if

                                                            yyxx

                                                            yx

                                                            yx

                                                            yxyxangle

                                                            tt

                                                            t

                                                            11 cos

                                                            cos

                                                            x y

                                                            yx 0 yx

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis

                                                            bull All Orthogonal to each other

                                                            ie for

                                                            bull All have Length 1

                                                            ie for

                                                            nvv 1

                                                            1 ii vv

                                                            0 ii vv ii

                                                            ni 1

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis (cont)

                                                            bull Spectral Representation

                                                            where

                                                            nvv 1

                                                            n

                                                            iii vax

                                                            1

                                                            ii vxa

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis (cont)

                                                            bull Spectral Representation

                                                            where

                                                            (Coefficient is Inner Product Cool Notation)

                                                            nvv 1

                                                            n

                                                            iii vax

                                                            1

                                                            ii vxa

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis (cont)

                                                            bull Spectral Representation

                                                            where

                                                            Check

                                                            nvv 1

                                                            n

                                                            iii vax

                                                            1

                                                            ii vxa

                                                            iii

                                                            n

                                                            iii

                                                            n

                                                            iiii avvavvavx

                                                            1

                                                            1

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis (cont)

                                                            bull Spectral Representation

                                                            where

                                                            Check

                                                            bull Matrix Notation where ie

                                                            For the Basis Matrix

                                                            nvv 1

                                                            n

                                                            iii vax

                                                            1

                                                            ii vxa

                                                            iii

                                                            n

                                                            iii

                                                            n

                                                            iiii avvavvavx

                                                            1

                                                            1

                                                            aBx Bxa tt xBa t

                                                            nvvB 1

                                                            Review of Linear Algebra (Cont)

                                                            Orthonormal Basis (cont)

                                                            bull Spectral Representation

                                                            where

                                                            Check

                                                            bull Matrix Notation where ie

                                                            is called transform of

                                                            (eg Fourier or Wavelet)

                                                            nvv 1

                                                            n

                                                            iii vax

                                                            1

                                                            ii vxa

                                                            iii

                                                            n

                                                            iii

                                                            n

                                                            iiii avvavvavx

                                                            1

                                                            1

                                                            aBx Bxa tt xBa t

                                                            xa

                                                            Review of Linear Algebra (Cont)

                                                            Parseval identity for

                                                            in subsp genrsquod by o n basis

                                                            x

                                                            nvv 1

                                                            2

                                                            1

                                                            22

                                                            1

                                                            2 aavxx

                                                            n

                                                            ii

                                                            n

                                                            ii

                                                            Review of Linear Algebra (Cont)

                                                            Parseval identity for

                                                            in subsp genrsquod by o n basis

                                                            bull Pythagorean theorem

                                                            bull ldquoDecomposition of Energyrdquo

                                                            bull ANOVA - sums of squares

                                                            x

                                                            nvv 1

                                                            2

                                                            1

                                                            22

                                                            1

                                                            2 aavxx

                                                            n

                                                            ii

                                                            n

                                                            ii

                                                            Review of Linear Algebra (Cont)

                                                            Parseval identity for

                                                            in subsp genrsquod by o n basis

                                                            bull Pythagorean theorem

                                                            bull ldquoDecomposition of Energyrdquo

                                                            bull ANOVA - sums of squares

                                                            bull Transform has same length as

                                                            ie ldquorotation in rdquo

                                                            x

                                                            nvv 1

                                                            2

                                                            1

                                                            22

                                                            1

                                                            2 aavxx

                                                            n

                                                            ii

                                                            n

                                                            ii

                                                            a xd

                                                            Projection of a Vector onto a Subspace

                                                            bull Idea Member of that is Closest to

                                                            (ie ldquoBest Approxrsquonrdquo)

                                                            Review of Linear Algebra (Cont)x

                                                            xV

                                                            V

                                                            Projection of a Vector onto a Subspace

                                                            bull Idea Member of that is Closest to

                                                            (ie ldquoBest Approxrsquonrdquo)

                                                            bull Find that Solves

                                                            (ldquoLeast Squaresrdquo)

                                                            Review of Linear Algebra (Cont)x

                                                            xV

                                                            V

                                                            VxPV vxVv

                                                            min

                                                            Projection of a Vector onto a Subspace

                                                            bull Idea Member of that is Closest to

                                                            (ie ldquoBest Approxrsquonrdquo)

                                                            bull Find that Solves

                                                            (ldquoLeast Squaresrdquo)

                                                            bull For Inner Product (Hilbert) Space

                                                            Exists and is Unique

                                                            Review of Linear Algebra (Cont)x

                                                            xV

                                                            V

                                                            VxPV vxVv

                                                            min

                                                            xPV

                                                            Projection of a Vector onto a Subspace (cont)

                                                            bull General Solution in for Basis Matrix

                                                            Review of Linear Algebra (Cont)

                                                            d VB

                                                            xBBBBxP tVV

                                                            tVVV

                                                            1

                                                            Projection of a Vector onto a Subspace (cont)

                                                            bull General Solution in for Basis Matrix

                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                            (thus projection is another linear operation)

                                                            Review of Linear Algebra (Cont)

                                                            d VB

                                                            xBBBBxP tVV

                                                            tVVV

                                                            1

                                                            tVV

                                                            tVVV BBBBP

                                                            1

                                                            Projection of a Vector onto a Subspace (cont)

                                                            bull General Solution in for Basis Matrix

                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                            (thus projection is another linear operation)

                                                            (note same operation underlies least squares)

                                                            Review of Linear Algebra (Cont)

                                                            d VB

                                                            xBBBBxP tVV

                                                            tVVV

                                                            1

                                                            tVV

                                                            tVVV BBBBP

                                                            1

                                                            Review of Linear Algebra (Cont)

                                                            Projection using Orthonormal Basis

                                                            bull Basis Matrix is Orthonormal

                                                            nnVtV IBB

                                                            10

                                                            01

                                                            1

                                                            111

                                                            1

                                                            1

                                                            nnn

                                                            n

                                                            ntn

                                                            t

                                                            vvvv

                                                            vvvv

                                                            vv

                                                            v

                                                            v

                                                            nvv 1

                                                            Review of Linear Algebra (Cont)

                                                            Projection using Orthonormal Basis

                                                            bull Basis Matrix is Orthonormal

                                                            bull So =

                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                            (Recall Right Multrsquon)

                                                            nnVtV IBB

                                                            10

                                                            01

                                                            1

                                                            111

                                                            1

                                                            1

                                                            nnn

                                                            n

                                                            ntn

                                                            t

                                                            vvvv

                                                            vvvv

                                                            vv

                                                            v

                                                            v

                                                            xBBxP tVVV

                                                            x V

                                                            nvv 1

                                                            Review of Linear Algebra (Cont)

                                                            Projection using Orthonormal Basis (cont)

                                                            bull For Orthogonal Complement

                                                            and

                                                            V

                                                            xPxPx VV 222xPxPx VV

                                                            Review of Linear Algebra (Cont)

                                                            Projection using Orthonormal Basis (cont)

                                                            bull For Orthogonal Complement

                                                            and

                                                            bull Parseval Inequality

                                                            V

                                                            xPxPx VV 222xPxPx VV

                                                            2

                                                            1

                                                            22

                                                            1

                                                            22 aavxxxP

                                                            n

                                                            ii

                                                            n

                                                            iiV

                                                            Review of Linear Algebra (Cont)

                                                            (Real) Unitary Matrices with

                                                            bull Orthonormal Basis Matrix

                                                            (So All of Above Applies)

                                                            ddU IUU t

                                                            Review of Linear Algebra (Cont)

                                                            (Real) Unitary Matrices with

                                                            bull Orthonormal Basis Matrix

                                                            (So All of Above Applies)

                                                            bull Note Transformrsquon is Distance Preserving

                                                            ddU IUU t

                                                            yxdyxyxyxUyUxUdn

                                                            i ii 2

                                                            1

                                                            Review of Linear Algebra (Cont)

                                                            (Real) Unitary Matrices with

                                                            bull Orthonormal Basis Matrix

                                                            (So All of Above Applies)

                                                            bull Note Transformrsquon is Distance Preserving

                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                            ddU IUU t

                                                            yxdyxyxyxUyUxUdn

                                                            i ii 2

                                                            1

                                                            U

                                                            Review of Linear Algebra (Cont)

                                                            Singular Value Decomposition (SVD)

                                                            For a Matrix

                                                            Find

                                                            ndX

                                                            Review of Linear Algebra (Cont)

                                                            Singular Value Decomposition (SVD)

                                                            For a Matrix

                                                            Find a Diagonal Matrix

                                                            with Entries

                                                            called Singular Values

                                                            ndX

                                                            ndS

                                                            )min(1 ndss

                                                            Review of Linear Algebra (Cont)

                                                            Singular Value Decomposition (SVD)

                                                            For a Matrix

                                                            Find a Diagonal Matrix

                                                            with Entries

                                                            called Singular Values

                                                            And Unitary (Rotation) Matrices

                                                            (recall )

                                                            ndX

                                                            ndS

                                                            )min(1 ndss

                                                            ddU nnV

                                                            IVVUU tt

                                                            Review of Linear Algebra (Cont)

                                                            Singular Value Decomposition (SVD)

                                                            For a Matrix

                                                            Find a Diagonal Matrix

                                                            with Entries

                                                            called Singular Values

                                                            And Unitary (Rotation) Matrices

                                                            (recall )

                                                            So That

                                                            ndX

                                                            ndS

                                                            )min(1 ndss

                                                            ddU nnV

                                                            IVVUU tt tUSVX

                                                            Review of Linear Algebra (Cont)

                                                            Intuition behind Singular Value Decomposition

                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                            vVSUvVSUvX tt

                                                            Review of Linear Algebra (Cont)

                                                            Intuition behind Singular Value Decomposition

                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                            bull First rotate

                                                            X

                                                            vVSUvVSUvX tt

                                                            Review of Linear Algebra (Cont)

                                                            Intuition behind Singular Value Decomposition

                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                            bull First rotate

                                                            bull Second rescale coordinate axes (by )

                                                            X

                                                            vVSUvVSUvX tt

                                                            is

                                                            Review of Linear Algebra (Cont)

                                                            Intuition behind Singular Value Decomposition

                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                            bull First rotate

                                                            bull Second rescale coordinate axes (by )

                                                            bull Third rotate again

                                                            X

                                                            vVSUvVSUvX tt

                                                            is

                                                            Review of Linear Algebra (Cont)

                                                            Intuition behind Singular Value Decomposition

                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                            bull First rotate

                                                            bull Second rescale coordinate axes (by )

                                                            bull Third rotate again

                                                            bull ie have diagonalized the transformation

                                                            X

                                                            vVSUvVSUvX tt

                                                            is

                                                            SVD Compact Representation

                                                            Useful Labeling

                                                            Singular Values in Increasing Order

                                                            Review of Linear Algebra (Cont)

                                                            )min(1 dnss

                                                            SVD Compact Representation

                                                            Useful Labeling

                                                            Singular Values in Increasing Order

                                                            Note singular values = 0 can be omitted

                                                            (Since do ldquo0-Stretchingrdquo)

                                                            Review of Linear Algebra (Cont)

                                                            )min(1 dnss

                                                            Review of Linear Algebra (Cont)

                                                            r

                                                            SVD Compact Representation

                                                            Useful Labeling

                                                            Singular Values in Increasing Order

                                                            Note singular values = 0 can be omitted

                                                            Let = of positive singular values

                                                            )min(1 dnss

                                                            Review of Linear Algebra (Cont)

                                                            r

                                                            SVD Compact Representation

                                                            Useful Labeling

                                                            Singular Values in Increasing Order

                                                            Note singular values = 0 can be omitted

                                                            Let = of positive singular values

                                                            Then

                                                            Where are truncations of

                                                            trnrrrd VSUX

                                                            VSU

                                                            )min(1 dnss

                                                            Review of Linear Algebra (Cont)

                                                            SVD Full Representation

                                                            =

                                                            Graphics Display Assumes

                                                            ndX ddU ndS nn

                                                            tV

                                                            nd

                                                            Review of Linear Algebra (Cont)

                                                            SVD Full Representation

                                                            =

                                                            Full Rank Basis Matrix

                                                            ndX ddU ndS nn

                                                            tV

                                                            Review of Linear Algebra (Cont)

                                                            SVD Full Representation

                                                            =

                                                            Full Rank Basis Matrix

                                                            All 0s in Bottom

                                                            ndX ddU ndS nn

                                                            tV

                                                            Review of Linear Algebra (Cont)

                                                            SVD Reduced Representation

                                                            =

                                                            These Columns Get 0ed Out

                                                            ndX ddU nnS nn

                                                            tV

                                                            nnd 0

                                                            Review of Linear Algebra (Cont)

                                                            SVD Reduced Representation

                                                            =ndX ndU nnS nn

                                                            tV

                                                            Review of Linear Algebra (Cont)

                                                            SVD Reduced Representation

                                                            =

                                                            Also Some of These May be 0

                                                            ndX ndU nnS nn

                                                            tV

                                                            Review of Linear Algebra (Cont)

                                                            SVD Compact Representation

                                                            = ndX rdU

                                                            rrS nrtV

                                                            0

                                                            Review of Linear Algebra (Cont)

                                                            SVD Compact Representation

                                                            =

                                                            These Get 0ed Out

                                                            ndX rdU

                                                            rrS nrtV

                                                            0

                                                            Review of Linear Algebra (Cont)

                                                            SVD Compact Representation

                                                            = ndX rdU

                                                            rrS nrtV

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition

                                                            For a (Symmetric) Square Matrix

                                                            Find

                                                            ddX

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition

                                                            For a (Symmetric) Square Matrix

                                                            Find a Diagonal Matrix

                                                            Called Eigenvalues

                                                            Convenient Ordering

                                                            ddX

                                                            d

                                                            D

                                                            0

                                                            01

                                                            n 1

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition

                                                            For a (Symmetric) Square Matrix

                                                            Find a Diagonal Matrix

                                                            And an Orthonormal Matrix

                                                            (ie )

                                                            ddX

                                                            d

                                                            D

                                                            0

                                                            01

                                                            ddB

                                                            ddtt IBBBB

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition

                                                            For a (Symmetric) Square Matrix

                                                            Find a Diagonal Matrix

                                                            And an Orthonormal Matrix

                                                            (ie )

                                                            So that ie

                                                            ddX

                                                            d

                                                            D

                                                            0

                                                            01

                                                            ddB

                                                            ddtt IBBBB

                                                            DBBX tBDBX

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                            (looks similar)

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                            Complex (uses )

                                                            VU

                                                            1i

                                                            Review of Linear Algebra (Cont)

                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                            VU

                                                            X

                                                            BVU

                                                            1i

                                                            • Participant Presentations
                                                            • PCA to find clusters
                                                            • Statistical Smoothing
                                                            • Kernel Density Estimation
                                                            • Scatterplot Smoothing
                                                            • Statistical Smoothing (2)
                                                            • SiZer Background
                                                            • SiZer Background (2)
                                                            • SiZer Background (3)
                                                            • SiZer Background (4)
                                                            • SiZer Background (5)
                                                            • SiZer Overview
                                                            • PCA to find clusters (2)
                                                            • PCA to find clusters (3)
                                                            • PCA to find clusters (4)
                                                            • PCA to find clusters (5)
                                                            • PCA to find clusters (6)
                                                            • PCA to find clusters (7)
                                                            • Recall Yeast Cell Cycle Data
                                                            • Yeast Cell Cycle Data FDA View
                                                            • Yeast Cell Cycle Data FDA View (2)
                                                            • Yeast Cell Cycle Data FDA View (3)
                                                            • Yeast Cell Cycles Freq 2 Proj
                                                            • Frequency 2 Analysis
                                                            • Frequency 2 Analysis (2)
                                                            • Frequency 2 Analysis (3)
                                                            • Yeast Cell Cycle
                                                            • SiZer Study of Distrsquon of Angles
                                                            • Reclassification of Major Genes
                                                            • Compare to Previous Classifrsquon
                                                            • New Subpopulation View
                                                            • New Subpopulation View (2)
                                                            • Detailed Look at PCA
                                                            • PCA Rediscovery ndash Renaming
                                                            • PCA Rediscovery ndash Renaming (2)
                                                            • PCA Rediscovery ndash Renaming (3)
                                                            • PCA Rediscovery ndash Renaming (4)
                                                            • PCA Rediscovery ndash Renaming (5)
                                                            • An Interesting Historical Note
                                                            • An Interesting Historical Note (2)
                                                            • Detailed Look at PCA (2)
                                                            • Detailed Look at PCA (3)
                                                            • Review of Linear Algebra
                                                            • Review of Linear Algebra (2)
                                                            • Review of Linear Algebra (3)
                                                            • Review of Linear Algebra (Cont)
                                                            • Review of Linear Algebra (Cont) (2)
                                                            • Review of Linear Algebra (Cont) (3)
                                                            • Review of Linear Algebra (Cont) (4)
                                                            • Review of Linear Algebra (Cont) (5)
                                                            • Review of Linear Algebra (Cont) (6)
                                                            • Review of Linear Algebra (Cont) (7)
                                                            • Review of Linear Algebra (Cont) (8)
                                                            • Review of Linear Algebra (Cont) (9)
                                                            • Review of Linear Algebra (Cont) (10)
                                                            • Review of Linear Algebra (Cont) (11)
                                                            • Review of Linear Algebra (Cont) (12)
                                                            • Review of Linear Algebra (Cont) (13)
                                                            • Review of Linear Algebra (Cont) (14)
                                                            • Review of Linear Algebra (Cont) (15)
                                                            • Review of Linear Algebra (Cont) (16)
                                                            • Review of Linear Algebra (Cont) (17)
                                                            • Review of Linear Algebra (Cont) (18)
                                                            • Review of Linear Algebra (Cont) (19)
                                                            • Review of Linear Algebra (Cont) (20)
                                                            • Review of Linear Algebra (Cont) (21)
                                                            • Review of Linear Algebra (Cont) (22)
                                                            • Review of Linear Algebra (Cont) (23)
                                                            • Review of Linear Algebra (Cont) (24)
                                                            • Review of Linear Algebra (Cont) (25)
                                                            • Review of Linear Algebra (Cont) (26)
                                                            • Review of Linear Algebra (Cont) (27)
                                                            • Review of Linear Algebra (Cont) (28)
                                                            • Review of Linear Algebra (Cont) (29)
                                                            • Review of Linear Algebra (Cont) (30)
                                                            • Review of Linear Algebra (Cont) (31)
                                                            • Review of Linear Algebra (Cont) (32)
                                                            • Review of Linear Algebra (Cont) (33)
                                                            • Review of Linear Algebra (Cont) (34)
                                                            • Review of Linear Algebra (Cont) (35)
                                                            • Review of Linear Algebra (Cont) (36)
                                                            • Review of Linear Algebra (Cont) (37)
                                                            • Review of Linear Algebra (Cont) (38)
                                                            • Review of Linear Algebra (Cont) (39)
                                                            • Review of Linear Algebra (Cont) (40)
                                                            • Review of Linear Algebra (Cont) (41)
                                                            • Review of Linear Algebra (Cont) (42)
                                                            • Review of Linear Algebra (Cont) (43)
                                                            • Review of Linear Algebra (Cont) (44)
                                                            • Review of Linear Algebra (Cont) (45)
                                                            • Review of Linear Algebra (Cont) (46)
                                                            • Review of Linear Algebra (Cont) (47)
                                                            • Review of Linear Algebra (Cont) (48)
                                                            • Review of Linear Algebra (Cont) (49)
                                                            • Review of Linear Algebra (Cont) (50)
                                                            • Review of Linear Algebra (Cont) (51)
                                                            • Review of Linear Algebra (Cont) (52)
                                                            • Review of Linear Algebra (Cont) (53)
                                                            • Review of Linear Algebra (Cont) (54)
                                                            • Review of Linear Algebra (Cont) (55)
                                                            • Review of Linear Algebra (Cont) (56)
                                                            • Review of Linear Algebra (Cont) (57)
                                                            • Review of Linear Algebra (Cont) (58)
                                                            • Review of Linear Algebra (Cont) (59)
                                                            • Review of Linear Algebra (Cont) (60)
                                                            • Review of Linear Algebra (Cont) (61)
                                                            • Review of Linear Algebra (Cont) (62)
                                                            • Review of Linear Algebra (Cont) (63)
                                                            • Review of Linear Algebra (Cont) (64)
                                                            • Review of Linear Algebra (Cont) (65)
                                                            • Review of Linear Algebra (Cont) (66)
                                                            • Review of Linear Algebra (Cont) (67)
                                                            • Review of Linear Algebra (Cont) (68)
                                                            • Review of Linear Algebra (Cont) (69)
                                                            • Review of Linear Algebra (Cont) (70)
                                                            • Review of Linear Algebra (Cont) (71)
                                                            • Review of Linear Algebra (Cont) (72)
                                                            • Review of Linear Algebra (Cont) (73)
                                                            • Review of Linear Algebra (Cont) (74)
                                                            • Review of Linear Algebra (Cont) (75)
                                                            • Review of Linear Algebra (Cont) (76)
                                                            • Review of Linear Algebra (Cont) (77)
                                                            • Review of Linear Algebra (Cont) (78)
                                                            • Review of Linear Algebra (Cont) (79)
                                                            • Review of Linear Algebra (Cont) (80)
                                                            • Review of Linear Algebra (Cont) (81)

                                                              New Subpopulation View

                                                              New Subpopulation View

                                                              NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                              Detailed Look at PCA

                                                              Now Study ldquoFolklorerdquo More Carefully

                                                              bull BackGround

                                                              bull History

                                                              bull Underpinnings

                                                              (Mathematical amp Computational)

                                                              Good Overall Reference Jolliffe (2002)

                                                              PCA Rediscovery ndash Renaming

                                                              Statistics Principal Component Analysis (PCA)

                                                              PCA Rediscovery ndash Renaming

                                                              Statistics Principal Component Analysis (PCA)

                                                              Social Sciences Factor Analysis (PCA is a subset)

                                                              PCA Rediscovery ndash Renaming

                                                              Statistics Principal Component Analysis (PCA)

                                                              Social Sciences Factor Analysis (PCA is a subset)

                                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                                              PCA Rediscovery ndash Renaming

                                                              Statistics Principal Component Analysis (PCA)

                                                              Social Sciences Factor Analysis (PCA is a subset)

                                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                                              Applied MathematicsProper Orthogonal Decomposition (POD)

                                                              PCA Rediscovery ndash Renaming

                                                              Statistics Principal Component Analysis (PCA)

                                                              Social Sciences Factor Analysis (PCA is a subset)

                                                              Probability Electrical EngKarhunen ndash Loeve expansion

                                                              Applied MathematicsProper Orthogonal Decomposition (POD)

                                                              Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                              An Interesting Historical Note

                                                              The 1st () application of PCA to Functional

                                                              Data Analysis

                                                              An Interesting Historical Note

                                                              The 1st () application of PCA to Functional

                                                              Data Analysis

                                                              Rao (1958)

                                                              1st Paper with ldquoCurves as Data Objectsrdquo

                                                              viewpoint

                                                              Detailed Look at PCA

                                                              Three Important (amp Interesting) Viewpoints

                                                              1 Mathematics

                                                              2 Numerics

                                                              3 Statistics

                                                              Goal Study Interrelationships

                                                              Detailed Look at PCA

                                                              Three Important (amp Interesting) Viewpoints

                                                              1 Mathematics

                                                              2 Numerics

                                                              3 Statistics

                                                              1st Review Linear Alg and Multivar Prob

                                                              Review of Linear Algebra

                                                              Vector Space

                                                              bull set of ldquovectorsrdquo

                                                              bull and ldquoscalarsrdquo (coefficients)

                                                              xa

                                                              Review of Linear Algebra

                                                              Vector Space

                                                              bull set of ldquovectorsrdquo

                                                              bull and ldquoscalarsrdquo (coefficients)

                                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                              ( in space)

                                                              xa

                                                              i

                                                              ii xa

                                                              Review of Linear Algebra

                                                              Vector Space

                                                              bull set of ldquovectorsrdquo

                                                              bull and ldquoscalarsrdquo (coefficients)

                                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                              ( in space)

                                                              eg

                                                              ldquo dim Euclidrsquon spacerdquo

                                                              xa

                                                              i

                                                              ii xa

                                                              d

                                                              d

                                                              d xx

                                                              x

                                                              x

                                                              x 1

                                                              1

                                                              d

                                                              Review of Linear Algebra (Cont)

                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                              Review of Linear Algebra (Cont)

                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                              Review of Linear Algebra (Cont)

                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                              Note Planes not Through the Origin

                                                              are not Subspaces

                                                              (Do not Contain )00 x

                                                              Review of Linear Algebra (Cont)

                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                              (all Linear Combos of them =

                                                              = Containing Hyperplane

                                                              through Origin)

                                                              Review of Linear Algebra (Cont)

                                                              Basis of Subspace Set of Vectors that

                                                              bull Span ie Everything is a Lin Com of them

                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                              Review of Linear Algebra (Cont)

                                                              Basis of Subspace Set of Vectors that

                                                              bull Span ie Everything is a Lin Com of them

                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                              bull eg ldquoUnit Vector Basisrdquo

                                                              bull Since

                                                              d

                                                              1

                                                              0

                                                              0

                                                              0

                                                              1

                                                              0

                                                              0

                                                              0

                                                              1

                                                              1

                                                              0

                                                              0

                                                              0

                                                              1

                                                              0

                                                              0

                                                              0

                                                              1

                                                              212

                                                              1

                                                              d

                                                              d

                                                              xxx

                                                              x

                                                              x

                                                              x

                                                              Review of Linear Algebra (Cont)

                                                              Basis Matrix of subspace of

                                                              Given a basis

                                                              create matrix of columns

                                                              dnvv 1

                                                              nddnd

                                                              n

                                                              n

                                                              vv

                                                              vv

                                                              vvB

                                                              1

                                                              111

                                                              1

                                                              Review of Linear Algebra (Cont)

                                                              Then linear combo is a matrix multiplicatrsquon

                                                              where

                                                              n

                                                              iii aBva

                                                              1

                                                              na

                                                              a

                                                              a 1

                                                              Review of Linear Algebra (Cont)

                                                              Then linear combo is a matrix multiplicatrsquon

                                                              where

                                                              Note Right Multiplication Gives

                                                              Linear Combination of Column Vectors

                                                              n

                                                              iii aBva

                                                              1

                                                              na

                                                              a

                                                              a 1

                                                              Review of Linear Algebra (Cont)

                                                              Then linear combo is a matrix multiplicatrsquon

                                                              where

                                                              Check sizes

                                                              n

                                                              iii aBva

                                                              1

                                                              na

                                                              a

                                                              a 1

                                                              )1()(1 nndd

                                                              Review of Linear Algebra (Cont)

                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                              For matrices

                                                              Define the Matrix Product

                                                              mkk

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              nmm

                                                              n

                                                              bb

                                                              bb

                                                              B

                                                              1

                                                              111

                                                              m

                                                              iniik

                                                              m

                                                              iiik

                                                              m

                                                              inii

                                                              m

                                                              iii

                                                              baba

                                                              baba

                                                              AB

                                                              1

                                                              11

                                                              11

                                                              111

                                                              Review of Linear Algebra (Cont)

                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                              For matrices

                                                              Define the Matrix Product

                                                              (Inner Products of Rows With Columns )

                                                              mkk

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              nmm

                                                              n

                                                              bb

                                                              bb

                                                              B

                                                              1

                                                              111

                                                              m

                                                              iniik

                                                              m

                                                              iiik

                                                              m

                                                              inii

                                                              m

                                                              iii

                                                              baba

                                                              baba

                                                              AB

                                                              1

                                                              11

                                                              11

                                                              111

                                                              A B

                                                              Review of Linear Algebra (Cont)

                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                              For matrices

                                                              Define the Matrix Product

                                                              (Inner Products of Rows With Columns )

                                                              (Composition of Linear Transformations)

                                                              mkk

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              nmm

                                                              n

                                                              bb

                                                              bb

                                                              B

                                                              1

                                                              111

                                                              m

                                                              iniik

                                                              m

                                                              iiik

                                                              m

                                                              inii

                                                              m

                                                              iii

                                                              baba

                                                              baba

                                                              AB

                                                              1

                                                              11

                                                              11

                                                              111

                                                              A B

                                                              Review of Linear Algebra (Cont)

                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                              For matrices

                                                              Define the Matrix Product

                                                              (Inner Products of Rows With Columns )

                                                              (Composition of Linear Transformations)

                                                              Often Useful to Check Sizes

                                                              mkk

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              nmm

                                                              n

                                                              bb

                                                              bb

                                                              B

                                                              1

                                                              111

                                                              m

                                                              iniik

                                                              m

                                                              iiik

                                                              m

                                                              inii

                                                              m

                                                              iii

                                                              baba

                                                              baba

                                                              AB

                                                              1

                                                              11

                                                              11

                                                              111

                                                              nmmknk

                                                              A B

                                                              Review of Linear Algebra (Cont)

                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                              For matrices

                                                              Define the Matrix Product

                                                              (Inner Products of Rows With Columns )

                                                              (Composition of Linear Transformations)

                                                              Often Useful to Check Sizes

                                                              mkk

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              nmm

                                                              n

                                                              bb

                                                              bb

                                                              B

                                                              1

                                                              111

                                                              m

                                                              iniik

                                                              m

                                                              iiik

                                                              m

                                                              inii

                                                              m

                                                              iii

                                                              baba

                                                              baba

                                                              AB

                                                              1

                                                              11

                                                              11

                                                              111

                                                              nmmknk

                                                              A B

                                                              Review of Linear Algebra (Cont)

                                                              Matrix Trace

                                                              bull For a Square Matrix

                                                              bull Define

                                                              mmm

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              m

                                                              iiiaAtr

                                                              1)(

                                                              Review of Linear Algebra (Cont)

                                                              Matrix Trace

                                                              bull For a Square Matrix

                                                              bull Define

                                                              bull Trace Commutes with Matrix Multiplication

                                                              mmm

                                                              m

                                                              aa

                                                              aa

                                                              A

                                                              1

                                                              111

                                                              m

                                                              iiiaAtr

                                                              1)(

                                                              BAtrABtr

                                                              Review of Linear Algebra (Cont)

                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                              bull Number of Elements in a Basis (Unique)

                                                              Review of Linear Algebra (Cont)

                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                              bull Number of Elements in a Basis (Unique)

                                                              bull (Use Basis Above)

                                                              bull eg dim of a line is 1

                                                              bull eg dim of a plane is 2

                                                              dd dim

                                                              Review of Linear Algebra (Cont)

                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                              bull Number of Elements in a Basis (Unique)

                                                              bull (Use Basis Above)

                                                              bull eg dim of a line is 1

                                                              bull eg dim of a plane is 2

                                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                                              (in Statistical Uses eg ANOVA)

                                                              dd dim

                                                              Review of Linear Algebra (Cont)

                                                              Norm of a Vector

                                                              bull in d 21

                                                              21

                                                              1

                                                              2 xxxx td

                                                              jj

                                                              Review of Linear Algebra (Cont)

                                                              Norm of a Vector

                                                              bull in

                                                              bull Idea length of the vector

                                                              d 2121

                                                              1

                                                              2 xxxx td

                                                              jj

                                                              Review of Linear Algebra (Cont)

                                                              Norm of a Vector

                                                              bull in

                                                              bull Idea length of the vector

                                                              bull Note strange properties for high

                                                              eg ldquolength of diagonal of unit cuberdquo =

                                                              d 2121

                                                              1

                                                              2 xxxx td

                                                              jj

                                                              d

                                                              d

                                                              Review of Linear Algebra (Cont)

                                                              Norm of a Vector (cont)

                                                              bull Length Normalized Vector

                                                              (has Length 1 thus on Surf of Unit Sphere

                                                              amp is a Direction Vector)

                                                              x

                                                              x

                                                              Review of Linear Algebra (Cont)

                                                              Norm of a Vector (cont)

                                                              bull Length Normalized Vector

                                                              (has Length 1 thus on Surf of Unit Sphere

                                                              amp is a Direction Vector)

                                                              bull Define Distance as

                                                              x

                                                              x

                                                              yxyxyxyxd t

                                                              Review of Linear Algebra (Cont)

                                                              Inner (Dot Scalar) Product

                                                              bull for Vectors and

                                                              yxyxyx td

                                                              jjj

                                                              1

                                                              x y

                                                              Review of Linear Algebra (Cont)

                                                              Inner (Dot Scalar) Product

                                                              bull for Vectors and

                                                              bull Related to Norm via

                                                              yxyxyx td

                                                              jjj

                                                              1

                                                              x y

                                                              21 xxx

                                                              Review of Linear Algebra (Cont)

                                                              Inner (Dot Scalar) Product (cont)

                                                              bull measures ldquoangle between and rdquo as

                                                              yyxx

                                                              yx

                                                              yx

                                                              yxyxangle

                                                              tt

                                                              t

                                                              11 cos

                                                              cos

                                                              x y

                                                              Review of Linear Algebra (Cont)

                                                              Inner (Dot Scalar) Product (cont)

                                                              bull measures ldquoangle between and rdquo as

                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                              if and only if

                                                              yyxx

                                                              yx

                                                              yx

                                                              yxyxangle

                                                              tt

                                                              t

                                                              11 cos

                                                              cos

                                                              x y

                                                              yx 0 yx

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis

                                                              bull All Orthogonal to each other

                                                              ie for

                                                              bull All have Length 1

                                                              ie for

                                                              nvv 1

                                                              1 ii vv

                                                              0 ii vv ii

                                                              ni 1

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis (cont)

                                                              bull Spectral Representation

                                                              where

                                                              nvv 1

                                                              n

                                                              iii vax

                                                              1

                                                              ii vxa

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis (cont)

                                                              bull Spectral Representation

                                                              where

                                                              (Coefficient is Inner Product Cool Notation)

                                                              nvv 1

                                                              n

                                                              iii vax

                                                              1

                                                              ii vxa

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis (cont)

                                                              bull Spectral Representation

                                                              where

                                                              Check

                                                              nvv 1

                                                              n

                                                              iii vax

                                                              1

                                                              ii vxa

                                                              iii

                                                              n

                                                              iii

                                                              n

                                                              iiii avvavvavx

                                                              1

                                                              1

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis (cont)

                                                              bull Spectral Representation

                                                              where

                                                              Check

                                                              bull Matrix Notation where ie

                                                              For the Basis Matrix

                                                              nvv 1

                                                              n

                                                              iii vax

                                                              1

                                                              ii vxa

                                                              iii

                                                              n

                                                              iii

                                                              n

                                                              iiii avvavvavx

                                                              1

                                                              1

                                                              aBx Bxa tt xBa t

                                                              nvvB 1

                                                              Review of Linear Algebra (Cont)

                                                              Orthonormal Basis (cont)

                                                              bull Spectral Representation

                                                              where

                                                              Check

                                                              bull Matrix Notation where ie

                                                              is called transform of

                                                              (eg Fourier or Wavelet)

                                                              nvv 1

                                                              n

                                                              iii vax

                                                              1

                                                              ii vxa

                                                              iii

                                                              n

                                                              iii

                                                              n

                                                              iiii avvavvavx

                                                              1

                                                              1

                                                              aBx Bxa tt xBa t

                                                              xa

                                                              Review of Linear Algebra (Cont)

                                                              Parseval identity for

                                                              in subsp genrsquod by o n basis

                                                              x

                                                              nvv 1

                                                              2

                                                              1

                                                              22

                                                              1

                                                              2 aavxx

                                                              n

                                                              ii

                                                              n

                                                              ii

                                                              Review of Linear Algebra (Cont)

                                                              Parseval identity for

                                                              in subsp genrsquod by o n basis

                                                              bull Pythagorean theorem

                                                              bull ldquoDecomposition of Energyrdquo

                                                              bull ANOVA - sums of squares

                                                              x

                                                              nvv 1

                                                              2

                                                              1

                                                              22

                                                              1

                                                              2 aavxx

                                                              n

                                                              ii

                                                              n

                                                              ii

                                                              Review of Linear Algebra (Cont)

                                                              Parseval identity for

                                                              in subsp genrsquod by o n basis

                                                              bull Pythagorean theorem

                                                              bull ldquoDecomposition of Energyrdquo

                                                              bull ANOVA - sums of squares

                                                              bull Transform has same length as

                                                              ie ldquorotation in rdquo

                                                              x

                                                              nvv 1

                                                              2

                                                              1

                                                              22

                                                              1

                                                              2 aavxx

                                                              n

                                                              ii

                                                              n

                                                              ii

                                                              a xd

                                                              Projection of a Vector onto a Subspace

                                                              bull Idea Member of that is Closest to

                                                              (ie ldquoBest Approxrsquonrdquo)

                                                              Review of Linear Algebra (Cont)x

                                                              xV

                                                              V

                                                              Projection of a Vector onto a Subspace

                                                              bull Idea Member of that is Closest to

                                                              (ie ldquoBest Approxrsquonrdquo)

                                                              bull Find that Solves

                                                              (ldquoLeast Squaresrdquo)

                                                              Review of Linear Algebra (Cont)x

                                                              xV

                                                              V

                                                              VxPV vxVv

                                                              min

                                                              Projection of a Vector onto a Subspace

                                                              bull Idea Member of that is Closest to

                                                              (ie ldquoBest Approxrsquonrdquo)

                                                              bull Find that Solves

                                                              (ldquoLeast Squaresrdquo)

                                                              bull For Inner Product (Hilbert) Space

                                                              Exists and is Unique

                                                              Review of Linear Algebra (Cont)x

                                                              xV

                                                              V

                                                              VxPV vxVv

                                                              min

                                                              xPV

                                                              Projection of a Vector onto a Subspace (cont)

                                                              bull General Solution in for Basis Matrix

                                                              Review of Linear Algebra (Cont)

                                                              d VB

                                                              xBBBBxP tVV

                                                              tVVV

                                                              1

                                                              Projection of a Vector onto a Subspace (cont)

                                                              bull General Solution in for Basis Matrix

                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                              (thus projection is another linear operation)

                                                              Review of Linear Algebra (Cont)

                                                              d VB

                                                              xBBBBxP tVV

                                                              tVVV

                                                              1

                                                              tVV

                                                              tVVV BBBBP

                                                              1

                                                              Projection of a Vector onto a Subspace (cont)

                                                              bull General Solution in for Basis Matrix

                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                              (thus projection is another linear operation)

                                                              (note same operation underlies least squares)

                                                              Review of Linear Algebra (Cont)

                                                              d VB

                                                              xBBBBxP tVV

                                                              tVVV

                                                              1

                                                              tVV

                                                              tVVV BBBBP

                                                              1

                                                              Review of Linear Algebra (Cont)

                                                              Projection using Orthonormal Basis

                                                              bull Basis Matrix is Orthonormal

                                                              nnVtV IBB

                                                              10

                                                              01

                                                              1

                                                              111

                                                              1

                                                              1

                                                              nnn

                                                              n

                                                              ntn

                                                              t

                                                              vvvv

                                                              vvvv

                                                              vv

                                                              v

                                                              v

                                                              nvv 1

                                                              Review of Linear Algebra (Cont)

                                                              Projection using Orthonormal Basis

                                                              bull Basis Matrix is Orthonormal

                                                              bull So =

                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                              (Recall Right Multrsquon)

                                                              nnVtV IBB

                                                              10

                                                              01

                                                              1

                                                              111

                                                              1

                                                              1

                                                              nnn

                                                              n

                                                              ntn

                                                              t

                                                              vvvv

                                                              vvvv

                                                              vv

                                                              v

                                                              v

                                                              xBBxP tVVV

                                                              x V

                                                              nvv 1

                                                              Review of Linear Algebra (Cont)

                                                              Projection using Orthonormal Basis (cont)

                                                              bull For Orthogonal Complement

                                                              and

                                                              V

                                                              xPxPx VV 222xPxPx VV

                                                              Review of Linear Algebra (Cont)

                                                              Projection using Orthonormal Basis (cont)

                                                              bull For Orthogonal Complement

                                                              and

                                                              bull Parseval Inequality

                                                              V

                                                              xPxPx VV 222xPxPx VV

                                                              2

                                                              1

                                                              22

                                                              1

                                                              22 aavxxxP

                                                              n

                                                              ii

                                                              n

                                                              iiV

                                                              Review of Linear Algebra (Cont)

                                                              (Real) Unitary Matrices with

                                                              bull Orthonormal Basis Matrix

                                                              (So All of Above Applies)

                                                              ddU IUU t

                                                              Review of Linear Algebra (Cont)

                                                              (Real) Unitary Matrices with

                                                              bull Orthonormal Basis Matrix

                                                              (So All of Above Applies)

                                                              bull Note Transformrsquon is Distance Preserving

                                                              ddU IUU t

                                                              yxdyxyxyxUyUxUdn

                                                              i ii 2

                                                              1

                                                              Review of Linear Algebra (Cont)

                                                              (Real) Unitary Matrices with

                                                              bull Orthonormal Basis Matrix

                                                              (So All of Above Applies)

                                                              bull Note Transformrsquon is Distance Preserving

                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                              ddU IUU t

                                                              yxdyxyxyxUyUxUdn

                                                              i ii 2

                                                              1

                                                              U

                                                              Review of Linear Algebra (Cont)

                                                              Singular Value Decomposition (SVD)

                                                              For a Matrix

                                                              Find

                                                              ndX

                                                              Review of Linear Algebra (Cont)

                                                              Singular Value Decomposition (SVD)

                                                              For a Matrix

                                                              Find a Diagonal Matrix

                                                              with Entries

                                                              called Singular Values

                                                              ndX

                                                              ndS

                                                              )min(1 ndss

                                                              Review of Linear Algebra (Cont)

                                                              Singular Value Decomposition (SVD)

                                                              For a Matrix

                                                              Find a Diagonal Matrix

                                                              with Entries

                                                              called Singular Values

                                                              And Unitary (Rotation) Matrices

                                                              (recall )

                                                              ndX

                                                              ndS

                                                              )min(1 ndss

                                                              ddU nnV

                                                              IVVUU tt

                                                              Review of Linear Algebra (Cont)

                                                              Singular Value Decomposition (SVD)

                                                              For a Matrix

                                                              Find a Diagonal Matrix

                                                              with Entries

                                                              called Singular Values

                                                              And Unitary (Rotation) Matrices

                                                              (recall )

                                                              So That

                                                              ndX

                                                              ndS

                                                              )min(1 ndss

                                                              ddU nnV

                                                              IVVUU tt tUSVX

                                                              Review of Linear Algebra (Cont)

                                                              Intuition behind Singular Value Decomposition

                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                              vVSUvVSUvX tt

                                                              Review of Linear Algebra (Cont)

                                                              Intuition behind Singular Value Decomposition

                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                              bull First rotate

                                                              X

                                                              vVSUvVSUvX tt

                                                              Review of Linear Algebra (Cont)

                                                              Intuition behind Singular Value Decomposition

                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                              bull First rotate

                                                              bull Second rescale coordinate axes (by )

                                                              X

                                                              vVSUvVSUvX tt

                                                              is

                                                              Review of Linear Algebra (Cont)

                                                              Intuition behind Singular Value Decomposition

                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                              bull First rotate

                                                              bull Second rescale coordinate axes (by )

                                                              bull Third rotate again

                                                              X

                                                              vVSUvVSUvX tt

                                                              is

                                                              Review of Linear Algebra (Cont)

                                                              Intuition behind Singular Value Decomposition

                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                              bull First rotate

                                                              bull Second rescale coordinate axes (by )

                                                              bull Third rotate again

                                                              bull ie have diagonalized the transformation

                                                              X

                                                              vVSUvVSUvX tt

                                                              is

                                                              SVD Compact Representation

                                                              Useful Labeling

                                                              Singular Values in Increasing Order

                                                              Review of Linear Algebra (Cont)

                                                              )min(1 dnss

                                                              SVD Compact Representation

                                                              Useful Labeling

                                                              Singular Values in Increasing Order

                                                              Note singular values = 0 can be omitted

                                                              (Since do ldquo0-Stretchingrdquo)

                                                              Review of Linear Algebra (Cont)

                                                              )min(1 dnss

                                                              Review of Linear Algebra (Cont)

                                                              r

                                                              SVD Compact Representation

                                                              Useful Labeling

                                                              Singular Values in Increasing Order

                                                              Note singular values = 0 can be omitted

                                                              Let = of positive singular values

                                                              )min(1 dnss

                                                              Review of Linear Algebra (Cont)

                                                              r

                                                              SVD Compact Representation

                                                              Useful Labeling

                                                              Singular Values in Increasing Order

                                                              Note singular values = 0 can be omitted

                                                              Let = of positive singular values

                                                              Then

                                                              Where are truncations of

                                                              trnrrrd VSUX

                                                              VSU

                                                              )min(1 dnss

                                                              Review of Linear Algebra (Cont)

                                                              SVD Full Representation

                                                              =

                                                              Graphics Display Assumes

                                                              ndX ddU ndS nn

                                                              tV

                                                              nd

                                                              Review of Linear Algebra (Cont)

                                                              SVD Full Representation

                                                              =

                                                              Full Rank Basis Matrix

                                                              ndX ddU ndS nn

                                                              tV

                                                              Review of Linear Algebra (Cont)

                                                              SVD Full Representation

                                                              =

                                                              Full Rank Basis Matrix

                                                              All 0s in Bottom

                                                              ndX ddU ndS nn

                                                              tV

                                                              Review of Linear Algebra (Cont)

                                                              SVD Reduced Representation

                                                              =

                                                              These Columns Get 0ed Out

                                                              ndX ddU nnS nn

                                                              tV

                                                              nnd 0

                                                              Review of Linear Algebra (Cont)

                                                              SVD Reduced Representation

                                                              =ndX ndU nnS nn

                                                              tV

                                                              Review of Linear Algebra (Cont)

                                                              SVD Reduced Representation

                                                              =

                                                              Also Some of These May be 0

                                                              ndX ndU nnS nn

                                                              tV

                                                              Review of Linear Algebra (Cont)

                                                              SVD Compact Representation

                                                              = ndX rdU

                                                              rrS nrtV

                                                              0

                                                              Review of Linear Algebra (Cont)

                                                              SVD Compact Representation

                                                              =

                                                              These Get 0ed Out

                                                              ndX rdU

                                                              rrS nrtV

                                                              0

                                                              Review of Linear Algebra (Cont)

                                                              SVD Compact Representation

                                                              = ndX rdU

                                                              rrS nrtV

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition

                                                              For a (Symmetric) Square Matrix

                                                              Find

                                                              ddX

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition

                                                              For a (Symmetric) Square Matrix

                                                              Find a Diagonal Matrix

                                                              Called Eigenvalues

                                                              Convenient Ordering

                                                              ddX

                                                              d

                                                              D

                                                              0

                                                              01

                                                              n 1

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition

                                                              For a (Symmetric) Square Matrix

                                                              Find a Diagonal Matrix

                                                              And an Orthonormal Matrix

                                                              (ie )

                                                              ddX

                                                              d

                                                              D

                                                              0

                                                              01

                                                              ddB

                                                              ddtt IBBBB

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition

                                                              For a (Symmetric) Square Matrix

                                                              Find a Diagonal Matrix

                                                              And an Orthonormal Matrix

                                                              (ie )

                                                              So that ie

                                                              ddX

                                                              d

                                                              D

                                                              0

                                                              01

                                                              ddB

                                                              ddtt IBBBB

                                                              DBBX tBDBX

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                              (looks similar)

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                              Complex (uses )

                                                              VU

                                                              1i

                                                              Review of Linear Algebra (Cont)

                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                              VU

                                                              X

                                                              BVU

                                                              1i

                                                              • Participant Presentations
                                                              • PCA to find clusters
                                                              • Statistical Smoothing
                                                              • Kernel Density Estimation
                                                              • Scatterplot Smoothing
                                                              • Statistical Smoothing (2)
                                                              • SiZer Background
                                                              • SiZer Background (2)
                                                              • SiZer Background (3)
                                                              • SiZer Background (4)
                                                              • SiZer Background (5)
                                                              • SiZer Overview
                                                              • PCA to find clusters (2)
                                                              • PCA to find clusters (3)
                                                              • PCA to find clusters (4)
                                                              • PCA to find clusters (5)
                                                              • PCA to find clusters (6)
                                                              • PCA to find clusters (7)
                                                              • Recall Yeast Cell Cycle Data
                                                              • Yeast Cell Cycle Data FDA View
                                                              • Yeast Cell Cycle Data FDA View (2)
                                                              • Yeast Cell Cycle Data FDA View (3)
                                                              • Yeast Cell Cycles Freq 2 Proj
                                                              • Frequency 2 Analysis
                                                              • Frequency 2 Analysis (2)
                                                              • Frequency 2 Analysis (3)
                                                              • Yeast Cell Cycle
                                                              • SiZer Study of Distrsquon of Angles
                                                              • Reclassification of Major Genes
                                                              • Compare to Previous Classifrsquon
                                                              • New Subpopulation View
                                                              • New Subpopulation View (2)
                                                              • Detailed Look at PCA
                                                              • PCA Rediscovery ndash Renaming
                                                              • PCA Rediscovery ndash Renaming (2)
                                                              • PCA Rediscovery ndash Renaming (3)
                                                              • PCA Rediscovery ndash Renaming (4)
                                                              • PCA Rediscovery ndash Renaming (5)
                                                              • An Interesting Historical Note
                                                              • An Interesting Historical Note (2)
                                                              • Detailed Look at PCA (2)
                                                              • Detailed Look at PCA (3)
                                                              • Review of Linear Algebra
                                                              • Review of Linear Algebra (2)
                                                              • Review of Linear Algebra (3)
                                                              • Review of Linear Algebra (Cont)
                                                              • Review of Linear Algebra (Cont) (2)
                                                              • Review of Linear Algebra (Cont) (3)
                                                              • Review of Linear Algebra (Cont) (4)
                                                              • Review of Linear Algebra (Cont) (5)
                                                              • Review of Linear Algebra (Cont) (6)
                                                              • Review of Linear Algebra (Cont) (7)
                                                              • Review of Linear Algebra (Cont) (8)
                                                              • Review of Linear Algebra (Cont) (9)
                                                              • Review of Linear Algebra (Cont) (10)
                                                              • Review of Linear Algebra (Cont) (11)
                                                              • Review of Linear Algebra (Cont) (12)
                                                              • Review of Linear Algebra (Cont) (13)
                                                              • Review of Linear Algebra (Cont) (14)
                                                              • Review of Linear Algebra (Cont) (15)
                                                              • Review of Linear Algebra (Cont) (16)
                                                              • Review of Linear Algebra (Cont) (17)
                                                              • Review of Linear Algebra (Cont) (18)
                                                              • Review of Linear Algebra (Cont) (19)
                                                              • Review of Linear Algebra (Cont) (20)
                                                              • Review of Linear Algebra (Cont) (21)
                                                              • Review of Linear Algebra (Cont) (22)
                                                              • Review of Linear Algebra (Cont) (23)
                                                              • Review of Linear Algebra (Cont) (24)
                                                              • Review of Linear Algebra (Cont) (25)
                                                              • Review of Linear Algebra (Cont) (26)
                                                              • Review of Linear Algebra (Cont) (27)
                                                              • Review of Linear Algebra (Cont) (28)
                                                              • Review of Linear Algebra (Cont) (29)
                                                              • Review of Linear Algebra (Cont) (30)
                                                              • Review of Linear Algebra (Cont) (31)
                                                              • Review of Linear Algebra (Cont) (32)
                                                              • Review of Linear Algebra (Cont) (33)
                                                              • Review of Linear Algebra (Cont) (34)
                                                              • Review of Linear Algebra (Cont) (35)
                                                              • Review of Linear Algebra (Cont) (36)
                                                              • Review of Linear Algebra (Cont) (37)
                                                              • Review of Linear Algebra (Cont) (38)
                                                              • Review of Linear Algebra (Cont) (39)
                                                              • Review of Linear Algebra (Cont) (40)
                                                              • Review of Linear Algebra (Cont) (41)
                                                              • Review of Linear Algebra (Cont) (42)
                                                              • Review of Linear Algebra (Cont) (43)
                                                              • Review of Linear Algebra (Cont) (44)
                                                              • Review of Linear Algebra (Cont) (45)
                                                              • Review of Linear Algebra (Cont) (46)
                                                              • Review of Linear Algebra (Cont) (47)
                                                              • Review of Linear Algebra (Cont) (48)
                                                              • Review of Linear Algebra (Cont) (49)
                                                              • Review of Linear Algebra (Cont) (50)
                                                              • Review of Linear Algebra (Cont) (51)
                                                              • Review of Linear Algebra (Cont) (52)
                                                              • Review of Linear Algebra (Cont) (53)
                                                              • Review of Linear Algebra (Cont) (54)
                                                              • Review of Linear Algebra (Cont) (55)
                                                              • Review of Linear Algebra (Cont) (56)
                                                              • Review of Linear Algebra (Cont) (57)
                                                              • Review of Linear Algebra (Cont) (58)
                                                              • Review of Linear Algebra (Cont) (59)
                                                              • Review of Linear Algebra (Cont) (60)
                                                              • Review of Linear Algebra (Cont) (61)
                                                              • Review of Linear Algebra (Cont) (62)
                                                              • Review of Linear Algebra (Cont) (63)
                                                              • Review of Linear Algebra (Cont) (64)
                                                              • Review of Linear Algebra (Cont) (65)
                                                              • Review of Linear Algebra (Cont) (66)
                                                              • Review of Linear Algebra (Cont) (67)
                                                              • Review of Linear Algebra (Cont) (68)
                                                              • Review of Linear Algebra (Cont) (69)
                                                              • Review of Linear Algebra (Cont) (70)
                                                              • Review of Linear Algebra (Cont) (71)
                                                              • Review of Linear Algebra (Cont) (72)
                                                              • Review of Linear Algebra (Cont) (73)
                                                              • Review of Linear Algebra (Cont) (74)
                                                              • Review of Linear Algebra (Cont) (75)
                                                              • Review of Linear Algebra (Cont) (76)
                                                              • Review of Linear Algebra (Cont) (77)
                                                              • Review of Linear Algebra (Cont) (78)
                                                              • Review of Linear Algebra (Cont) (79)
                                                              • Review of Linear Algebra (Cont) (80)
                                                              • Review of Linear Algebra (Cont) (81)

                                                                New Subpopulation View

                                                                NoteSubdensitiesHave SameBandwidth ampProportionalAreas(so Σ = 1)

                                                                Detailed Look at PCA

                                                                Now Study ldquoFolklorerdquo More Carefully

                                                                bull BackGround

                                                                bull History

                                                                bull Underpinnings

                                                                (Mathematical amp Computational)

                                                                Good Overall Reference Jolliffe (2002)

                                                                PCA Rediscovery ndash Renaming

                                                                Statistics Principal Component Analysis (PCA)

                                                                PCA Rediscovery ndash Renaming

                                                                Statistics Principal Component Analysis (PCA)

                                                                Social Sciences Factor Analysis (PCA is a subset)

                                                                PCA Rediscovery ndash Renaming

                                                                Statistics Principal Component Analysis (PCA)

                                                                Social Sciences Factor Analysis (PCA is a subset)

                                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                                PCA Rediscovery ndash Renaming

                                                                Statistics Principal Component Analysis (PCA)

                                                                Social Sciences Factor Analysis (PCA is a subset)

                                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                PCA Rediscovery ndash Renaming

                                                                Statistics Principal Component Analysis (PCA)

                                                                Social Sciences Factor Analysis (PCA is a subset)

                                                                Probability Electrical EngKarhunen ndash Loeve expansion

                                                                Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                An Interesting Historical Note

                                                                The 1st () application of PCA to Functional

                                                                Data Analysis

                                                                An Interesting Historical Note

                                                                The 1st () application of PCA to Functional

                                                                Data Analysis

                                                                Rao (1958)

                                                                1st Paper with ldquoCurves as Data Objectsrdquo

                                                                viewpoint

                                                                Detailed Look at PCA

                                                                Three Important (amp Interesting) Viewpoints

                                                                1 Mathematics

                                                                2 Numerics

                                                                3 Statistics

                                                                Goal Study Interrelationships

                                                                Detailed Look at PCA

                                                                Three Important (amp Interesting) Viewpoints

                                                                1 Mathematics

                                                                2 Numerics

                                                                3 Statistics

                                                                1st Review Linear Alg and Multivar Prob

                                                                Review of Linear Algebra

                                                                Vector Space

                                                                bull set of ldquovectorsrdquo

                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                xa

                                                                Review of Linear Algebra

                                                                Vector Space

                                                                bull set of ldquovectorsrdquo

                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                ( in space)

                                                                xa

                                                                i

                                                                ii xa

                                                                Review of Linear Algebra

                                                                Vector Space

                                                                bull set of ldquovectorsrdquo

                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                ( in space)

                                                                eg

                                                                ldquo dim Euclidrsquon spacerdquo

                                                                xa

                                                                i

                                                                ii xa

                                                                d

                                                                d

                                                                d xx

                                                                x

                                                                x

                                                                x 1

                                                                1

                                                                d

                                                                Review of Linear Algebra (Cont)

                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                Review of Linear Algebra (Cont)

                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                Review of Linear Algebra (Cont)

                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                Note Planes not Through the Origin

                                                                are not Subspaces

                                                                (Do not Contain )00 x

                                                                Review of Linear Algebra (Cont)

                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                (all Linear Combos of them =

                                                                = Containing Hyperplane

                                                                through Origin)

                                                                Review of Linear Algebra (Cont)

                                                                Basis of Subspace Set of Vectors that

                                                                bull Span ie Everything is a Lin Com of them

                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                Review of Linear Algebra (Cont)

                                                                Basis of Subspace Set of Vectors that

                                                                bull Span ie Everything is a Lin Com of them

                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                bull eg ldquoUnit Vector Basisrdquo

                                                                bull Since

                                                                d

                                                                1

                                                                0

                                                                0

                                                                0

                                                                1

                                                                0

                                                                0

                                                                0

                                                                1

                                                                1

                                                                0

                                                                0

                                                                0

                                                                1

                                                                0

                                                                0

                                                                0

                                                                1

                                                                212

                                                                1

                                                                d

                                                                d

                                                                xxx

                                                                x

                                                                x

                                                                x

                                                                Review of Linear Algebra (Cont)

                                                                Basis Matrix of subspace of

                                                                Given a basis

                                                                create matrix of columns

                                                                dnvv 1

                                                                nddnd

                                                                n

                                                                n

                                                                vv

                                                                vv

                                                                vvB

                                                                1

                                                                111

                                                                1

                                                                Review of Linear Algebra (Cont)

                                                                Then linear combo is a matrix multiplicatrsquon

                                                                where

                                                                n

                                                                iii aBva

                                                                1

                                                                na

                                                                a

                                                                a 1

                                                                Review of Linear Algebra (Cont)

                                                                Then linear combo is a matrix multiplicatrsquon

                                                                where

                                                                Note Right Multiplication Gives

                                                                Linear Combination of Column Vectors

                                                                n

                                                                iii aBva

                                                                1

                                                                na

                                                                a

                                                                a 1

                                                                Review of Linear Algebra (Cont)

                                                                Then linear combo is a matrix multiplicatrsquon

                                                                where

                                                                Check sizes

                                                                n

                                                                iii aBva

                                                                1

                                                                na

                                                                a

                                                                a 1

                                                                )1()(1 nndd

                                                                Review of Linear Algebra (Cont)

                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                For matrices

                                                                Define the Matrix Product

                                                                mkk

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                nmm

                                                                n

                                                                bb

                                                                bb

                                                                B

                                                                1

                                                                111

                                                                m

                                                                iniik

                                                                m

                                                                iiik

                                                                m

                                                                inii

                                                                m

                                                                iii

                                                                baba

                                                                baba

                                                                AB

                                                                1

                                                                11

                                                                11

                                                                111

                                                                Review of Linear Algebra (Cont)

                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                For matrices

                                                                Define the Matrix Product

                                                                (Inner Products of Rows With Columns )

                                                                mkk

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                nmm

                                                                n

                                                                bb

                                                                bb

                                                                B

                                                                1

                                                                111

                                                                m

                                                                iniik

                                                                m

                                                                iiik

                                                                m

                                                                inii

                                                                m

                                                                iii

                                                                baba

                                                                baba

                                                                AB

                                                                1

                                                                11

                                                                11

                                                                111

                                                                A B

                                                                Review of Linear Algebra (Cont)

                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                For matrices

                                                                Define the Matrix Product

                                                                (Inner Products of Rows With Columns )

                                                                (Composition of Linear Transformations)

                                                                mkk

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                nmm

                                                                n

                                                                bb

                                                                bb

                                                                B

                                                                1

                                                                111

                                                                m

                                                                iniik

                                                                m

                                                                iiik

                                                                m

                                                                inii

                                                                m

                                                                iii

                                                                baba

                                                                baba

                                                                AB

                                                                1

                                                                11

                                                                11

                                                                111

                                                                A B

                                                                Review of Linear Algebra (Cont)

                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                For matrices

                                                                Define the Matrix Product

                                                                (Inner Products of Rows With Columns )

                                                                (Composition of Linear Transformations)

                                                                Often Useful to Check Sizes

                                                                mkk

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                nmm

                                                                n

                                                                bb

                                                                bb

                                                                B

                                                                1

                                                                111

                                                                m

                                                                iniik

                                                                m

                                                                iiik

                                                                m

                                                                inii

                                                                m

                                                                iii

                                                                baba

                                                                baba

                                                                AB

                                                                1

                                                                11

                                                                11

                                                                111

                                                                nmmknk

                                                                A B

                                                                Review of Linear Algebra (Cont)

                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                For matrices

                                                                Define the Matrix Product

                                                                (Inner Products of Rows With Columns )

                                                                (Composition of Linear Transformations)

                                                                Often Useful to Check Sizes

                                                                mkk

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                nmm

                                                                n

                                                                bb

                                                                bb

                                                                B

                                                                1

                                                                111

                                                                m

                                                                iniik

                                                                m

                                                                iiik

                                                                m

                                                                inii

                                                                m

                                                                iii

                                                                baba

                                                                baba

                                                                AB

                                                                1

                                                                11

                                                                11

                                                                111

                                                                nmmknk

                                                                A B

                                                                Review of Linear Algebra (Cont)

                                                                Matrix Trace

                                                                bull For a Square Matrix

                                                                bull Define

                                                                mmm

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                m

                                                                iiiaAtr

                                                                1)(

                                                                Review of Linear Algebra (Cont)

                                                                Matrix Trace

                                                                bull For a Square Matrix

                                                                bull Define

                                                                bull Trace Commutes with Matrix Multiplication

                                                                mmm

                                                                m

                                                                aa

                                                                aa

                                                                A

                                                                1

                                                                111

                                                                m

                                                                iiiaAtr

                                                                1)(

                                                                BAtrABtr

                                                                Review of Linear Algebra (Cont)

                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                bull Number of Elements in a Basis (Unique)

                                                                Review of Linear Algebra (Cont)

                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                bull Number of Elements in a Basis (Unique)

                                                                bull (Use Basis Above)

                                                                bull eg dim of a line is 1

                                                                bull eg dim of a plane is 2

                                                                dd dim

                                                                Review of Linear Algebra (Cont)

                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                bull Number of Elements in a Basis (Unique)

                                                                bull (Use Basis Above)

                                                                bull eg dim of a line is 1

                                                                bull eg dim of a plane is 2

                                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                                (in Statistical Uses eg ANOVA)

                                                                dd dim

                                                                Review of Linear Algebra (Cont)

                                                                Norm of a Vector

                                                                bull in d 21

                                                                21

                                                                1

                                                                2 xxxx td

                                                                jj

                                                                Review of Linear Algebra (Cont)

                                                                Norm of a Vector

                                                                bull in

                                                                bull Idea length of the vector

                                                                d 2121

                                                                1

                                                                2 xxxx td

                                                                jj

                                                                Review of Linear Algebra (Cont)

                                                                Norm of a Vector

                                                                bull in

                                                                bull Idea length of the vector

                                                                bull Note strange properties for high

                                                                eg ldquolength of diagonal of unit cuberdquo =

                                                                d 2121

                                                                1

                                                                2 xxxx td

                                                                jj

                                                                d

                                                                d

                                                                Review of Linear Algebra (Cont)

                                                                Norm of a Vector (cont)

                                                                bull Length Normalized Vector

                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                amp is a Direction Vector)

                                                                x

                                                                x

                                                                Review of Linear Algebra (Cont)

                                                                Norm of a Vector (cont)

                                                                bull Length Normalized Vector

                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                amp is a Direction Vector)

                                                                bull Define Distance as

                                                                x

                                                                x

                                                                yxyxyxyxd t

                                                                Review of Linear Algebra (Cont)

                                                                Inner (Dot Scalar) Product

                                                                bull for Vectors and

                                                                yxyxyx td

                                                                jjj

                                                                1

                                                                x y

                                                                Review of Linear Algebra (Cont)

                                                                Inner (Dot Scalar) Product

                                                                bull for Vectors and

                                                                bull Related to Norm via

                                                                yxyxyx td

                                                                jjj

                                                                1

                                                                x y

                                                                21 xxx

                                                                Review of Linear Algebra (Cont)

                                                                Inner (Dot Scalar) Product (cont)

                                                                bull measures ldquoangle between and rdquo as

                                                                yyxx

                                                                yx

                                                                yx

                                                                yxyxangle

                                                                tt

                                                                t

                                                                11 cos

                                                                cos

                                                                x y

                                                                Review of Linear Algebra (Cont)

                                                                Inner (Dot Scalar) Product (cont)

                                                                bull measures ldquoangle between and rdquo as

                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                if and only if

                                                                yyxx

                                                                yx

                                                                yx

                                                                yxyxangle

                                                                tt

                                                                t

                                                                11 cos

                                                                cos

                                                                x y

                                                                yx 0 yx

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis

                                                                bull All Orthogonal to each other

                                                                ie for

                                                                bull All have Length 1

                                                                ie for

                                                                nvv 1

                                                                1 ii vv

                                                                0 ii vv ii

                                                                ni 1

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis (cont)

                                                                bull Spectral Representation

                                                                where

                                                                nvv 1

                                                                n

                                                                iii vax

                                                                1

                                                                ii vxa

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis (cont)

                                                                bull Spectral Representation

                                                                where

                                                                (Coefficient is Inner Product Cool Notation)

                                                                nvv 1

                                                                n

                                                                iii vax

                                                                1

                                                                ii vxa

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis (cont)

                                                                bull Spectral Representation

                                                                where

                                                                Check

                                                                nvv 1

                                                                n

                                                                iii vax

                                                                1

                                                                ii vxa

                                                                iii

                                                                n

                                                                iii

                                                                n

                                                                iiii avvavvavx

                                                                1

                                                                1

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis (cont)

                                                                bull Spectral Representation

                                                                where

                                                                Check

                                                                bull Matrix Notation where ie

                                                                For the Basis Matrix

                                                                nvv 1

                                                                n

                                                                iii vax

                                                                1

                                                                ii vxa

                                                                iii

                                                                n

                                                                iii

                                                                n

                                                                iiii avvavvavx

                                                                1

                                                                1

                                                                aBx Bxa tt xBa t

                                                                nvvB 1

                                                                Review of Linear Algebra (Cont)

                                                                Orthonormal Basis (cont)

                                                                bull Spectral Representation

                                                                where

                                                                Check

                                                                bull Matrix Notation where ie

                                                                is called transform of

                                                                (eg Fourier or Wavelet)

                                                                nvv 1

                                                                n

                                                                iii vax

                                                                1

                                                                ii vxa

                                                                iii

                                                                n

                                                                iii

                                                                n

                                                                iiii avvavvavx

                                                                1

                                                                1

                                                                aBx Bxa tt xBa t

                                                                xa

                                                                Review of Linear Algebra (Cont)

                                                                Parseval identity for

                                                                in subsp genrsquod by o n basis

                                                                x

                                                                nvv 1

                                                                2

                                                                1

                                                                22

                                                                1

                                                                2 aavxx

                                                                n

                                                                ii

                                                                n

                                                                ii

                                                                Review of Linear Algebra (Cont)

                                                                Parseval identity for

                                                                in subsp genrsquod by o n basis

                                                                bull Pythagorean theorem

                                                                bull ldquoDecomposition of Energyrdquo

                                                                bull ANOVA - sums of squares

                                                                x

                                                                nvv 1

                                                                2

                                                                1

                                                                22

                                                                1

                                                                2 aavxx

                                                                n

                                                                ii

                                                                n

                                                                ii

                                                                Review of Linear Algebra (Cont)

                                                                Parseval identity for

                                                                in subsp genrsquod by o n basis

                                                                bull Pythagorean theorem

                                                                bull ldquoDecomposition of Energyrdquo

                                                                bull ANOVA - sums of squares

                                                                bull Transform has same length as

                                                                ie ldquorotation in rdquo

                                                                x

                                                                nvv 1

                                                                2

                                                                1

                                                                22

                                                                1

                                                                2 aavxx

                                                                n

                                                                ii

                                                                n

                                                                ii

                                                                a xd

                                                                Projection of a Vector onto a Subspace

                                                                bull Idea Member of that is Closest to

                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                Review of Linear Algebra (Cont)x

                                                                xV

                                                                V

                                                                Projection of a Vector onto a Subspace

                                                                bull Idea Member of that is Closest to

                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                bull Find that Solves

                                                                (ldquoLeast Squaresrdquo)

                                                                Review of Linear Algebra (Cont)x

                                                                xV

                                                                V

                                                                VxPV vxVv

                                                                min

                                                                Projection of a Vector onto a Subspace

                                                                bull Idea Member of that is Closest to

                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                bull Find that Solves

                                                                (ldquoLeast Squaresrdquo)

                                                                bull For Inner Product (Hilbert) Space

                                                                Exists and is Unique

                                                                Review of Linear Algebra (Cont)x

                                                                xV

                                                                V

                                                                VxPV vxVv

                                                                min

                                                                xPV

                                                                Projection of a Vector onto a Subspace (cont)

                                                                bull General Solution in for Basis Matrix

                                                                Review of Linear Algebra (Cont)

                                                                d VB

                                                                xBBBBxP tVV

                                                                tVVV

                                                                1

                                                                Projection of a Vector onto a Subspace (cont)

                                                                bull General Solution in for Basis Matrix

                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                (thus projection is another linear operation)

                                                                Review of Linear Algebra (Cont)

                                                                d VB

                                                                xBBBBxP tVV

                                                                tVVV

                                                                1

                                                                tVV

                                                                tVVV BBBBP

                                                                1

                                                                Projection of a Vector onto a Subspace (cont)

                                                                bull General Solution in for Basis Matrix

                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                (thus projection is another linear operation)

                                                                (note same operation underlies least squares)

                                                                Review of Linear Algebra (Cont)

                                                                d VB

                                                                xBBBBxP tVV

                                                                tVVV

                                                                1

                                                                tVV

                                                                tVVV BBBBP

                                                                1

                                                                Review of Linear Algebra (Cont)

                                                                Projection using Orthonormal Basis

                                                                bull Basis Matrix is Orthonormal

                                                                nnVtV IBB

                                                                10

                                                                01

                                                                1

                                                                111

                                                                1

                                                                1

                                                                nnn

                                                                n

                                                                ntn

                                                                t

                                                                vvvv

                                                                vvvv

                                                                vv

                                                                v

                                                                v

                                                                nvv 1

                                                                Review of Linear Algebra (Cont)

                                                                Projection using Orthonormal Basis

                                                                bull Basis Matrix is Orthonormal

                                                                bull So =

                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                (Recall Right Multrsquon)

                                                                nnVtV IBB

                                                                10

                                                                01

                                                                1

                                                                111

                                                                1

                                                                1

                                                                nnn

                                                                n

                                                                ntn

                                                                t

                                                                vvvv

                                                                vvvv

                                                                vv

                                                                v

                                                                v

                                                                xBBxP tVVV

                                                                x V

                                                                nvv 1

                                                                Review of Linear Algebra (Cont)

                                                                Projection using Orthonormal Basis (cont)

                                                                bull For Orthogonal Complement

                                                                and

                                                                V

                                                                xPxPx VV 222xPxPx VV

                                                                Review of Linear Algebra (Cont)

                                                                Projection using Orthonormal Basis (cont)

                                                                bull For Orthogonal Complement

                                                                and

                                                                bull Parseval Inequality

                                                                V

                                                                xPxPx VV 222xPxPx VV

                                                                2

                                                                1

                                                                22

                                                                1

                                                                22 aavxxxP

                                                                n

                                                                ii

                                                                n

                                                                iiV

                                                                Review of Linear Algebra (Cont)

                                                                (Real) Unitary Matrices with

                                                                bull Orthonormal Basis Matrix

                                                                (So All of Above Applies)

                                                                ddU IUU t

                                                                Review of Linear Algebra (Cont)

                                                                (Real) Unitary Matrices with

                                                                bull Orthonormal Basis Matrix

                                                                (So All of Above Applies)

                                                                bull Note Transformrsquon is Distance Preserving

                                                                ddU IUU t

                                                                yxdyxyxyxUyUxUdn

                                                                i ii 2

                                                                1

                                                                Review of Linear Algebra (Cont)

                                                                (Real) Unitary Matrices with

                                                                bull Orthonormal Basis Matrix

                                                                (So All of Above Applies)

                                                                bull Note Transformrsquon is Distance Preserving

                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                ddU IUU t

                                                                yxdyxyxyxUyUxUdn

                                                                i ii 2

                                                                1

                                                                U

                                                                Review of Linear Algebra (Cont)

                                                                Singular Value Decomposition (SVD)

                                                                For a Matrix

                                                                Find

                                                                ndX

                                                                Review of Linear Algebra (Cont)

                                                                Singular Value Decomposition (SVD)

                                                                For a Matrix

                                                                Find a Diagonal Matrix

                                                                with Entries

                                                                called Singular Values

                                                                ndX

                                                                ndS

                                                                )min(1 ndss

                                                                Review of Linear Algebra (Cont)

                                                                Singular Value Decomposition (SVD)

                                                                For a Matrix

                                                                Find a Diagonal Matrix

                                                                with Entries

                                                                called Singular Values

                                                                And Unitary (Rotation) Matrices

                                                                (recall )

                                                                ndX

                                                                ndS

                                                                )min(1 ndss

                                                                ddU nnV

                                                                IVVUU tt

                                                                Review of Linear Algebra (Cont)

                                                                Singular Value Decomposition (SVD)

                                                                For a Matrix

                                                                Find a Diagonal Matrix

                                                                with Entries

                                                                called Singular Values

                                                                And Unitary (Rotation) Matrices

                                                                (recall )

                                                                So That

                                                                ndX

                                                                ndS

                                                                )min(1 ndss

                                                                ddU nnV

                                                                IVVUU tt tUSVX

                                                                Review of Linear Algebra (Cont)

                                                                Intuition behind Singular Value Decomposition

                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                vVSUvVSUvX tt

                                                                Review of Linear Algebra (Cont)

                                                                Intuition behind Singular Value Decomposition

                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                bull First rotate

                                                                X

                                                                vVSUvVSUvX tt

                                                                Review of Linear Algebra (Cont)

                                                                Intuition behind Singular Value Decomposition

                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                bull First rotate

                                                                bull Second rescale coordinate axes (by )

                                                                X

                                                                vVSUvVSUvX tt

                                                                is

                                                                Review of Linear Algebra (Cont)

                                                                Intuition behind Singular Value Decomposition

                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                bull First rotate

                                                                bull Second rescale coordinate axes (by )

                                                                bull Third rotate again

                                                                X

                                                                vVSUvVSUvX tt

                                                                is

                                                                Review of Linear Algebra (Cont)

                                                                Intuition behind Singular Value Decomposition

                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                bull First rotate

                                                                bull Second rescale coordinate axes (by )

                                                                bull Third rotate again

                                                                bull ie have diagonalized the transformation

                                                                X

                                                                vVSUvVSUvX tt

                                                                is

                                                                SVD Compact Representation

                                                                Useful Labeling

                                                                Singular Values in Increasing Order

                                                                Review of Linear Algebra (Cont)

                                                                )min(1 dnss

                                                                SVD Compact Representation

                                                                Useful Labeling

                                                                Singular Values in Increasing Order

                                                                Note singular values = 0 can be omitted

                                                                (Since do ldquo0-Stretchingrdquo)

                                                                Review of Linear Algebra (Cont)

                                                                )min(1 dnss

                                                                Review of Linear Algebra (Cont)

                                                                r

                                                                SVD Compact Representation

                                                                Useful Labeling

                                                                Singular Values in Increasing Order

                                                                Note singular values = 0 can be omitted

                                                                Let = of positive singular values

                                                                )min(1 dnss

                                                                Review of Linear Algebra (Cont)

                                                                r

                                                                SVD Compact Representation

                                                                Useful Labeling

                                                                Singular Values in Increasing Order

                                                                Note singular values = 0 can be omitted

                                                                Let = of positive singular values

                                                                Then

                                                                Where are truncations of

                                                                trnrrrd VSUX

                                                                VSU

                                                                )min(1 dnss

                                                                Review of Linear Algebra (Cont)

                                                                SVD Full Representation

                                                                =

                                                                Graphics Display Assumes

                                                                ndX ddU ndS nn

                                                                tV

                                                                nd

                                                                Review of Linear Algebra (Cont)

                                                                SVD Full Representation

                                                                =

                                                                Full Rank Basis Matrix

                                                                ndX ddU ndS nn

                                                                tV

                                                                Review of Linear Algebra (Cont)

                                                                SVD Full Representation

                                                                =

                                                                Full Rank Basis Matrix

                                                                All 0s in Bottom

                                                                ndX ddU ndS nn

                                                                tV

                                                                Review of Linear Algebra (Cont)

                                                                SVD Reduced Representation

                                                                =

                                                                These Columns Get 0ed Out

                                                                ndX ddU nnS nn

                                                                tV

                                                                nnd 0

                                                                Review of Linear Algebra (Cont)

                                                                SVD Reduced Representation

                                                                =ndX ndU nnS nn

                                                                tV

                                                                Review of Linear Algebra (Cont)

                                                                SVD Reduced Representation

                                                                =

                                                                Also Some of These May be 0

                                                                ndX ndU nnS nn

                                                                tV

                                                                Review of Linear Algebra (Cont)

                                                                SVD Compact Representation

                                                                = ndX rdU

                                                                rrS nrtV

                                                                0

                                                                Review of Linear Algebra (Cont)

                                                                SVD Compact Representation

                                                                =

                                                                These Get 0ed Out

                                                                ndX rdU

                                                                rrS nrtV

                                                                0

                                                                Review of Linear Algebra (Cont)

                                                                SVD Compact Representation

                                                                = ndX rdU

                                                                rrS nrtV

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition

                                                                For a (Symmetric) Square Matrix

                                                                Find

                                                                ddX

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition

                                                                For a (Symmetric) Square Matrix

                                                                Find a Diagonal Matrix

                                                                Called Eigenvalues

                                                                Convenient Ordering

                                                                ddX

                                                                d

                                                                D

                                                                0

                                                                01

                                                                n 1

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition

                                                                For a (Symmetric) Square Matrix

                                                                Find a Diagonal Matrix

                                                                And an Orthonormal Matrix

                                                                (ie )

                                                                ddX

                                                                d

                                                                D

                                                                0

                                                                01

                                                                ddB

                                                                ddtt IBBBB

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition

                                                                For a (Symmetric) Square Matrix

                                                                Find a Diagonal Matrix

                                                                And an Orthonormal Matrix

                                                                (ie )

                                                                So that ie

                                                                ddX

                                                                d

                                                                D

                                                                0

                                                                01

                                                                ddB

                                                                ddtt IBBBB

                                                                DBBX tBDBX

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                (looks similar)

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                Complex (uses )

                                                                VU

                                                                1i

                                                                Review of Linear Algebra (Cont)

                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                VU

                                                                X

                                                                BVU

                                                                1i

                                                                • Participant Presentations
                                                                • PCA to find clusters
                                                                • Statistical Smoothing
                                                                • Kernel Density Estimation
                                                                • Scatterplot Smoothing
                                                                • Statistical Smoothing (2)
                                                                • SiZer Background
                                                                • SiZer Background (2)
                                                                • SiZer Background (3)
                                                                • SiZer Background (4)
                                                                • SiZer Background (5)
                                                                • SiZer Overview
                                                                • PCA to find clusters (2)
                                                                • PCA to find clusters (3)
                                                                • PCA to find clusters (4)
                                                                • PCA to find clusters (5)
                                                                • PCA to find clusters (6)
                                                                • PCA to find clusters (7)
                                                                • Recall Yeast Cell Cycle Data
                                                                • Yeast Cell Cycle Data FDA View
                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                • Frequency 2 Analysis
                                                                • Frequency 2 Analysis (2)
                                                                • Frequency 2 Analysis (3)
                                                                • Yeast Cell Cycle
                                                                • SiZer Study of Distrsquon of Angles
                                                                • Reclassification of Major Genes
                                                                • Compare to Previous Classifrsquon
                                                                • New Subpopulation View
                                                                • New Subpopulation View (2)
                                                                • Detailed Look at PCA
                                                                • PCA Rediscovery ndash Renaming
                                                                • PCA Rediscovery ndash Renaming (2)
                                                                • PCA Rediscovery ndash Renaming (3)
                                                                • PCA Rediscovery ndash Renaming (4)
                                                                • PCA Rediscovery ndash Renaming (5)
                                                                • An Interesting Historical Note
                                                                • An Interesting Historical Note (2)
                                                                • Detailed Look at PCA (2)
                                                                • Detailed Look at PCA (3)
                                                                • Review of Linear Algebra
                                                                • Review of Linear Algebra (2)
                                                                • Review of Linear Algebra (3)
                                                                • Review of Linear Algebra (Cont)
                                                                • Review of Linear Algebra (Cont) (2)
                                                                • Review of Linear Algebra (Cont) (3)
                                                                • Review of Linear Algebra (Cont) (4)
                                                                • Review of Linear Algebra (Cont) (5)
                                                                • Review of Linear Algebra (Cont) (6)
                                                                • Review of Linear Algebra (Cont) (7)
                                                                • Review of Linear Algebra (Cont) (8)
                                                                • Review of Linear Algebra (Cont) (9)
                                                                • Review of Linear Algebra (Cont) (10)
                                                                • Review of Linear Algebra (Cont) (11)
                                                                • Review of Linear Algebra (Cont) (12)
                                                                • Review of Linear Algebra (Cont) (13)
                                                                • Review of Linear Algebra (Cont) (14)
                                                                • Review of Linear Algebra (Cont) (15)
                                                                • Review of Linear Algebra (Cont) (16)
                                                                • Review of Linear Algebra (Cont) (17)
                                                                • Review of Linear Algebra (Cont) (18)
                                                                • Review of Linear Algebra (Cont) (19)
                                                                • Review of Linear Algebra (Cont) (20)
                                                                • Review of Linear Algebra (Cont) (21)
                                                                • Review of Linear Algebra (Cont) (22)
                                                                • Review of Linear Algebra (Cont) (23)
                                                                • Review of Linear Algebra (Cont) (24)
                                                                • Review of Linear Algebra (Cont) (25)
                                                                • Review of Linear Algebra (Cont) (26)
                                                                • Review of Linear Algebra (Cont) (27)
                                                                • Review of Linear Algebra (Cont) (28)
                                                                • Review of Linear Algebra (Cont) (29)
                                                                • Review of Linear Algebra (Cont) (30)
                                                                • Review of Linear Algebra (Cont) (31)
                                                                • Review of Linear Algebra (Cont) (32)
                                                                • Review of Linear Algebra (Cont) (33)
                                                                • Review of Linear Algebra (Cont) (34)
                                                                • Review of Linear Algebra (Cont) (35)
                                                                • Review of Linear Algebra (Cont) (36)
                                                                • Review of Linear Algebra (Cont) (37)
                                                                • Review of Linear Algebra (Cont) (38)
                                                                • Review of Linear Algebra (Cont) (39)
                                                                • Review of Linear Algebra (Cont) (40)
                                                                • Review of Linear Algebra (Cont) (41)
                                                                • Review of Linear Algebra (Cont) (42)
                                                                • Review of Linear Algebra (Cont) (43)
                                                                • Review of Linear Algebra (Cont) (44)
                                                                • Review of Linear Algebra (Cont) (45)
                                                                • Review of Linear Algebra (Cont) (46)
                                                                • Review of Linear Algebra (Cont) (47)
                                                                • Review of Linear Algebra (Cont) (48)
                                                                • Review of Linear Algebra (Cont) (49)
                                                                • Review of Linear Algebra (Cont) (50)
                                                                • Review of Linear Algebra (Cont) (51)
                                                                • Review of Linear Algebra (Cont) (52)
                                                                • Review of Linear Algebra (Cont) (53)
                                                                • Review of Linear Algebra (Cont) (54)
                                                                • Review of Linear Algebra (Cont) (55)
                                                                • Review of Linear Algebra (Cont) (56)
                                                                • Review of Linear Algebra (Cont) (57)
                                                                • Review of Linear Algebra (Cont) (58)
                                                                • Review of Linear Algebra (Cont) (59)
                                                                • Review of Linear Algebra (Cont) (60)
                                                                • Review of Linear Algebra (Cont) (61)
                                                                • Review of Linear Algebra (Cont) (62)
                                                                • Review of Linear Algebra (Cont) (63)
                                                                • Review of Linear Algebra (Cont) (64)
                                                                • Review of Linear Algebra (Cont) (65)
                                                                • Review of Linear Algebra (Cont) (66)
                                                                • Review of Linear Algebra (Cont) (67)
                                                                • Review of Linear Algebra (Cont) (68)
                                                                • Review of Linear Algebra (Cont) (69)
                                                                • Review of Linear Algebra (Cont) (70)
                                                                • Review of Linear Algebra (Cont) (71)
                                                                • Review of Linear Algebra (Cont) (72)
                                                                • Review of Linear Algebra (Cont) (73)
                                                                • Review of Linear Algebra (Cont) (74)
                                                                • Review of Linear Algebra (Cont) (75)
                                                                • Review of Linear Algebra (Cont) (76)
                                                                • Review of Linear Algebra (Cont) (77)
                                                                • Review of Linear Algebra (Cont) (78)
                                                                • Review of Linear Algebra (Cont) (79)
                                                                • Review of Linear Algebra (Cont) (80)
                                                                • Review of Linear Algebra (Cont) (81)

                                                                  Detailed Look at PCA

                                                                  Now Study ldquoFolklorerdquo More Carefully

                                                                  bull BackGround

                                                                  bull History

                                                                  bull Underpinnings

                                                                  (Mathematical amp Computational)

                                                                  Good Overall Reference Jolliffe (2002)

                                                                  PCA Rediscovery ndash Renaming

                                                                  Statistics Principal Component Analysis (PCA)

                                                                  PCA Rediscovery ndash Renaming

                                                                  Statistics Principal Component Analysis (PCA)

                                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                                  PCA Rediscovery ndash Renaming

                                                                  Statistics Principal Component Analysis (PCA)

                                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                                  PCA Rediscovery ndash Renaming

                                                                  Statistics Principal Component Analysis (PCA)

                                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                  PCA Rediscovery ndash Renaming

                                                                  Statistics Principal Component Analysis (PCA)

                                                                  Social Sciences Factor Analysis (PCA is a subset)

                                                                  Probability Electrical EngKarhunen ndash Loeve expansion

                                                                  Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                  Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                  An Interesting Historical Note

                                                                  The 1st () application of PCA to Functional

                                                                  Data Analysis

                                                                  An Interesting Historical Note

                                                                  The 1st () application of PCA to Functional

                                                                  Data Analysis

                                                                  Rao (1958)

                                                                  1st Paper with ldquoCurves as Data Objectsrdquo

                                                                  viewpoint

                                                                  Detailed Look at PCA

                                                                  Three Important (amp Interesting) Viewpoints

                                                                  1 Mathematics

                                                                  2 Numerics

                                                                  3 Statistics

                                                                  Goal Study Interrelationships

                                                                  Detailed Look at PCA

                                                                  Three Important (amp Interesting) Viewpoints

                                                                  1 Mathematics

                                                                  2 Numerics

                                                                  3 Statistics

                                                                  1st Review Linear Alg and Multivar Prob

                                                                  Review of Linear Algebra

                                                                  Vector Space

                                                                  bull set of ldquovectorsrdquo

                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                  xa

                                                                  Review of Linear Algebra

                                                                  Vector Space

                                                                  bull set of ldquovectorsrdquo

                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                  ( in space)

                                                                  xa

                                                                  i

                                                                  ii xa

                                                                  Review of Linear Algebra

                                                                  Vector Space

                                                                  bull set of ldquovectorsrdquo

                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                  ( in space)

                                                                  eg

                                                                  ldquo dim Euclidrsquon spacerdquo

                                                                  xa

                                                                  i

                                                                  ii xa

                                                                  d

                                                                  d

                                                                  d xx

                                                                  x

                                                                  x

                                                                  x 1

                                                                  1

                                                                  d

                                                                  Review of Linear Algebra (Cont)

                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                  Review of Linear Algebra (Cont)

                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                  Review of Linear Algebra (Cont)

                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                  Note Planes not Through the Origin

                                                                  are not Subspaces

                                                                  (Do not Contain )00 x

                                                                  Review of Linear Algebra (Cont)

                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                  (all Linear Combos of them =

                                                                  = Containing Hyperplane

                                                                  through Origin)

                                                                  Review of Linear Algebra (Cont)

                                                                  Basis of Subspace Set of Vectors that

                                                                  bull Span ie Everything is a Lin Com of them

                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                  Review of Linear Algebra (Cont)

                                                                  Basis of Subspace Set of Vectors that

                                                                  bull Span ie Everything is a Lin Com of them

                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                  bull eg ldquoUnit Vector Basisrdquo

                                                                  bull Since

                                                                  d

                                                                  1

                                                                  0

                                                                  0

                                                                  0

                                                                  1

                                                                  0

                                                                  0

                                                                  0

                                                                  1

                                                                  1

                                                                  0

                                                                  0

                                                                  0

                                                                  1

                                                                  0

                                                                  0

                                                                  0

                                                                  1

                                                                  212

                                                                  1

                                                                  d

                                                                  d

                                                                  xxx

                                                                  x

                                                                  x

                                                                  x

                                                                  Review of Linear Algebra (Cont)

                                                                  Basis Matrix of subspace of

                                                                  Given a basis

                                                                  create matrix of columns

                                                                  dnvv 1

                                                                  nddnd

                                                                  n

                                                                  n

                                                                  vv

                                                                  vv

                                                                  vvB

                                                                  1

                                                                  111

                                                                  1

                                                                  Review of Linear Algebra (Cont)

                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                  where

                                                                  n

                                                                  iii aBva

                                                                  1

                                                                  na

                                                                  a

                                                                  a 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                  where

                                                                  Note Right Multiplication Gives

                                                                  Linear Combination of Column Vectors

                                                                  n

                                                                  iii aBva

                                                                  1

                                                                  na

                                                                  a

                                                                  a 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                  where

                                                                  Check sizes

                                                                  n

                                                                  iii aBva

                                                                  1

                                                                  na

                                                                  a

                                                                  a 1

                                                                  )1()(1 nndd

                                                                  Review of Linear Algebra (Cont)

                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                  For matrices

                                                                  Define the Matrix Product

                                                                  mkk

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  nmm

                                                                  n

                                                                  bb

                                                                  bb

                                                                  B

                                                                  1

                                                                  111

                                                                  m

                                                                  iniik

                                                                  m

                                                                  iiik

                                                                  m

                                                                  inii

                                                                  m

                                                                  iii

                                                                  baba

                                                                  baba

                                                                  AB

                                                                  1

                                                                  11

                                                                  11

                                                                  111

                                                                  Review of Linear Algebra (Cont)

                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                  For matrices

                                                                  Define the Matrix Product

                                                                  (Inner Products of Rows With Columns )

                                                                  mkk

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  nmm

                                                                  n

                                                                  bb

                                                                  bb

                                                                  B

                                                                  1

                                                                  111

                                                                  m

                                                                  iniik

                                                                  m

                                                                  iiik

                                                                  m

                                                                  inii

                                                                  m

                                                                  iii

                                                                  baba

                                                                  baba

                                                                  AB

                                                                  1

                                                                  11

                                                                  11

                                                                  111

                                                                  A B

                                                                  Review of Linear Algebra (Cont)

                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                  For matrices

                                                                  Define the Matrix Product

                                                                  (Inner Products of Rows With Columns )

                                                                  (Composition of Linear Transformations)

                                                                  mkk

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  nmm

                                                                  n

                                                                  bb

                                                                  bb

                                                                  B

                                                                  1

                                                                  111

                                                                  m

                                                                  iniik

                                                                  m

                                                                  iiik

                                                                  m

                                                                  inii

                                                                  m

                                                                  iii

                                                                  baba

                                                                  baba

                                                                  AB

                                                                  1

                                                                  11

                                                                  11

                                                                  111

                                                                  A B

                                                                  Review of Linear Algebra (Cont)

                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                  For matrices

                                                                  Define the Matrix Product

                                                                  (Inner Products of Rows With Columns )

                                                                  (Composition of Linear Transformations)

                                                                  Often Useful to Check Sizes

                                                                  mkk

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  nmm

                                                                  n

                                                                  bb

                                                                  bb

                                                                  B

                                                                  1

                                                                  111

                                                                  m

                                                                  iniik

                                                                  m

                                                                  iiik

                                                                  m

                                                                  inii

                                                                  m

                                                                  iii

                                                                  baba

                                                                  baba

                                                                  AB

                                                                  1

                                                                  11

                                                                  11

                                                                  111

                                                                  nmmknk

                                                                  A B

                                                                  Review of Linear Algebra (Cont)

                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                  For matrices

                                                                  Define the Matrix Product

                                                                  (Inner Products of Rows With Columns )

                                                                  (Composition of Linear Transformations)

                                                                  Often Useful to Check Sizes

                                                                  mkk

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  nmm

                                                                  n

                                                                  bb

                                                                  bb

                                                                  B

                                                                  1

                                                                  111

                                                                  m

                                                                  iniik

                                                                  m

                                                                  iiik

                                                                  m

                                                                  inii

                                                                  m

                                                                  iii

                                                                  baba

                                                                  baba

                                                                  AB

                                                                  1

                                                                  11

                                                                  11

                                                                  111

                                                                  nmmknk

                                                                  A B

                                                                  Review of Linear Algebra (Cont)

                                                                  Matrix Trace

                                                                  bull For a Square Matrix

                                                                  bull Define

                                                                  mmm

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  m

                                                                  iiiaAtr

                                                                  1)(

                                                                  Review of Linear Algebra (Cont)

                                                                  Matrix Trace

                                                                  bull For a Square Matrix

                                                                  bull Define

                                                                  bull Trace Commutes with Matrix Multiplication

                                                                  mmm

                                                                  m

                                                                  aa

                                                                  aa

                                                                  A

                                                                  1

                                                                  111

                                                                  m

                                                                  iiiaAtr

                                                                  1)(

                                                                  BAtrABtr

                                                                  Review of Linear Algebra (Cont)

                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                  bull Number of Elements in a Basis (Unique)

                                                                  Review of Linear Algebra (Cont)

                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                  bull Number of Elements in a Basis (Unique)

                                                                  bull (Use Basis Above)

                                                                  bull eg dim of a line is 1

                                                                  bull eg dim of a plane is 2

                                                                  dd dim

                                                                  Review of Linear Algebra (Cont)

                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                  bull Number of Elements in a Basis (Unique)

                                                                  bull (Use Basis Above)

                                                                  bull eg dim of a line is 1

                                                                  bull eg dim of a plane is 2

                                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                                  (in Statistical Uses eg ANOVA)

                                                                  dd dim

                                                                  Review of Linear Algebra (Cont)

                                                                  Norm of a Vector

                                                                  bull in d 21

                                                                  21

                                                                  1

                                                                  2 xxxx td

                                                                  jj

                                                                  Review of Linear Algebra (Cont)

                                                                  Norm of a Vector

                                                                  bull in

                                                                  bull Idea length of the vector

                                                                  d 2121

                                                                  1

                                                                  2 xxxx td

                                                                  jj

                                                                  Review of Linear Algebra (Cont)

                                                                  Norm of a Vector

                                                                  bull in

                                                                  bull Idea length of the vector

                                                                  bull Note strange properties for high

                                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                                  d 2121

                                                                  1

                                                                  2 xxxx td

                                                                  jj

                                                                  d

                                                                  d

                                                                  Review of Linear Algebra (Cont)

                                                                  Norm of a Vector (cont)

                                                                  bull Length Normalized Vector

                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                  amp is a Direction Vector)

                                                                  x

                                                                  x

                                                                  Review of Linear Algebra (Cont)

                                                                  Norm of a Vector (cont)

                                                                  bull Length Normalized Vector

                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                  amp is a Direction Vector)

                                                                  bull Define Distance as

                                                                  x

                                                                  x

                                                                  yxyxyxyxd t

                                                                  Review of Linear Algebra (Cont)

                                                                  Inner (Dot Scalar) Product

                                                                  bull for Vectors and

                                                                  yxyxyx td

                                                                  jjj

                                                                  1

                                                                  x y

                                                                  Review of Linear Algebra (Cont)

                                                                  Inner (Dot Scalar) Product

                                                                  bull for Vectors and

                                                                  bull Related to Norm via

                                                                  yxyxyx td

                                                                  jjj

                                                                  1

                                                                  x y

                                                                  21 xxx

                                                                  Review of Linear Algebra (Cont)

                                                                  Inner (Dot Scalar) Product (cont)

                                                                  bull measures ldquoangle between and rdquo as

                                                                  yyxx

                                                                  yx

                                                                  yx

                                                                  yxyxangle

                                                                  tt

                                                                  t

                                                                  11 cos

                                                                  cos

                                                                  x y

                                                                  Review of Linear Algebra (Cont)

                                                                  Inner (Dot Scalar) Product (cont)

                                                                  bull measures ldquoangle between and rdquo as

                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                  if and only if

                                                                  yyxx

                                                                  yx

                                                                  yx

                                                                  yxyxangle

                                                                  tt

                                                                  t

                                                                  11 cos

                                                                  cos

                                                                  x y

                                                                  yx 0 yx

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis

                                                                  bull All Orthogonal to each other

                                                                  ie for

                                                                  bull All have Length 1

                                                                  ie for

                                                                  nvv 1

                                                                  1 ii vv

                                                                  0 ii vv ii

                                                                  ni 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis (cont)

                                                                  bull Spectral Representation

                                                                  where

                                                                  nvv 1

                                                                  n

                                                                  iii vax

                                                                  1

                                                                  ii vxa

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis (cont)

                                                                  bull Spectral Representation

                                                                  where

                                                                  (Coefficient is Inner Product Cool Notation)

                                                                  nvv 1

                                                                  n

                                                                  iii vax

                                                                  1

                                                                  ii vxa

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis (cont)

                                                                  bull Spectral Representation

                                                                  where

                                                                  Check

                                                                  nvv 1

                                                                  n

                                                                  iii vax

                                                                  1

                                                                  ii vxa

                                                                  iii

                                                                  n

                                                                  iii

                                                                  n

                                                                  iiii avvavvavx

                                                                  1

                                                                  1

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis (cont)

                                                                  bull Spectral Representation

                                                                  where

                                                                  Check

                                                                  bull Matrix Notation where ie

                                                                  For the Basis Matrix

                                                                  nvv 1

                                                                  n

                                                                  iii vax

                                                                  1

                                                                  ii vxa

                                                                  iii

                                                                  n

                                                                  iii

                                                                  n

                                                                  iiii avvavvavx

                                                                  1

                                                                  1

                                                                  aBx Bxa tt xBa t

                                                                  nvvB 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Orthonormal Basis (cont)

                                                                  bull Spectral Representation

                                                                  where

                                                                  Check

                                                                  bull Matrix Notation where ie

                                                                  is called transform of

                                                                  (eg Fourier or Wavelet)

                                                                  nvv 1

                                                                  n

                                                                  iii vax

                                                                  1

                                                                  ii vxa

                                                                  iii

                                                                  n

                                                                  iii

                                                                  n

                                                                  iiii avvavvavx

                                                                  1

                                                                  1

                                                                  aBx Bxa tt xBa t

                                                                  xa

                                                                  Review of Linear Algebra (Cont)

                                                                  Parseval identity for

                                                                  in subsp genrsquod by o n basis

                                                                  x

                                                                  nvv 1

                                                                  2

                                                                  1

                                                                  22

                                                                  1

                                                                  2 aavxx

                                                                  n

                                                                  ii

                                                                  n

                                                                  ii

                                                                  Review of Linear Algebra (Cont)

                                                                  Parseval identity for

                                                                  in subsp genrsquod by o n basis

                                                                  bull Pythagorean theorem

                                                                  bull ldquoDecomposition of Energyrdquo

                                                                  bull ANOVA - sums of squares

                                                                  x

                                                                  nvv 1

                                                                  2

                                                                  1

                                                                  22

                                                                  1

                                                                  2 aavxx

                                                                  n

                                                                  ii

                                                                  n

                                                                  ii

                                                                  Review of Linear Algebra (Cont)

                                                                  Parseval identity for

                                                                  in subsp genrsquod by o n basis

                                                                  bull Pythagorean theorem

                                                                  bull ldquoDecomposition of Energyrdquo

                                                                  bull ANOVA - sums of squares

                                                                  bull Transform has same length as

                                                                  ie ldquorotation in rdquo

                                                                  x

                                                                  nvv 1

                                                                  2

                                                                  1

                                                                  22

                                                                  1

                                                                  2 aavxx

                                                                  n

                                                                  ii

                                                                  n

                                                                  ii

                                                                  a xd

                                                                  Projection of a Vector onto a Subspace

                                                                  bull Idea Member of that is Closest to

                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                  Review of Linear Algebra (Cont)x

                                                                  xV

                                                                  V

                                                                  Projection of a Vector onto a Subspace

                                                                  bull Idea Member of that is Closest to

                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                  bull Find that Solves

                                                                  (ldquoLeast Squaresrdquo)

                                                                  Review of Linear Algebra (Cont)x

                                                                  xV

                                                                  V

                                                                  VxPV vxVv

                                                                  min

                                                                  Projection of a Vector onto a Subspace

                                                                  bull Idea Member of that is Closest to

                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                  bull Find that Solves

                                                                  (ldquoLeast Squaresrdquo)

                                                                  bull For Inner Product (Hilbert) Space

                                                                  Exists and is Unique

                                                                  Review of Linear Algebra (Cont)x

                                                                  xV

                                                                  V

                                                                  VxPV vxVv

                                                                  min

                                                                  xPV

                                                                  Projection of a Vector onto a Subspace (cont)

                                                                  bull General Solution in for Basis Matrix

                                                                  Review of Linear Algebra (Cont)

                                                                  d VB

                                                                  xBBBBxP tVV

                                                                  tVVV

                                                                  1

                                                                  Projection of a Vector onto a Subspace (cont)

                                                                  bull General Solution in for Basis Matrix

                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                  (thus projection is another linear operation)

                                                                  Review of Linear Algebra (Cont)

                                                                  d VB

                                                                  xBBBBxP tVV

                                                                  tVVV

                                                                  1

                                                                  tVV

                                                                  tVVV BBBBP

                                                                  1

                                                                  Projection of a Vector onto a Subspace (cont)

                                                                  bull General Solution in for Basis Matrix

                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                  (thus projection is another linear operation)

                                                                  (note same operation underlies least squares)

                                                                  Review of Linear Algebra (Cont)

                                                                  d VB

                                                                  xBBBBxP tVV

                                                                  tVVV

                                                                  1

                                                                  tVV

                                                                  tVVV BBBBP

                                                                  1

                                                                  Review of Linear Algebra (Cont)

                                                                  Projection using Orthonormal Basis

                                                                  bull Basis Matrix is Orthonormal

                                                                  nnVtV IBB

                                                                  10

                                                                  01

                                                                  1

                                                                  111

                                                                  1

                                                                  1

                                                                  nnn

                                                                  n

                                                                  ntn

                                                                  t

                                                                  vvvv

                                                                  vvvv

                                                                  vv

                                                                  v

                                                                  v

                                                                  nvv 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Projection using Orthonormal Basis

                                                                  bull Basis Matrix is Orthonormal

                                                                  bull So =

                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                  (Recall Right Multrsquon)

                                                                  nnVtV IBB

                                                                  10

                                                                  01

                                                                  1

                                                                  111

                                                                  1

                                                                  1

                                                                  nnn

                                                                  n

                                                                  ntn

                                                                  t

                                                                  vvvv

                                                                  vvvv

                                                                  vv

                                                                  v

                                                                  v

                                                                  xBBxP tVVV

                                                                  x V

                                                                  nvv 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Projection using Orthonormal Basis (cont)

                                                                  bull For Orthogonal Complement

                                                                  and

                                                                  V

                                                                  xPxPx VV 222xPxPx VV

                                                                  Review of Linear Algebra (Cont)

                                                                  Projection using Orthonormal Basis (cont)

                                                                  bull For Orthogonal Complement

                                                                  and

                                                                  bull Parseval Inequality

                                                                  V

                                                                  xPxPx VV 222xPxPx VV

                                                                  2

                                                                  1

                                                                  22

                                                                  1

                                                                  22 aavxxxP

                                                                  n

                                                                  ii

                                                                  n

                                                                  iiV

                                                                  Review of Linear Algebra (Cont)

                                                                  (Real) Unitary Matrices with

                                                                  bull Orthonormal Basis Matrix

                                                                  (So All of Above Applies)

                                                                  ddU IUU t

                                                                  Review of Linear Algebra (Cont)

                                                                  (Real) Unitary Matrices with

                                                                  bull Orthonormal Basis Matrix

                                                                  (So All of Above Applies)

                                                                  bull Note Transformrsquon is Distance Preserving

                                                                  ddU IUU t

                                                                  yxdyxyxyxUyUxUdn

                                                                  i ii 2

                                                                  1

                                                                  Review of Linear Algebra (Cont)

                                                                  (Real) Unitary Matrices with

                                                                  bull Orthonormal Basis Matrix

                                                                  (So All of Above Applies)

                                                                  bull Note Transformrsquon is Distance Preserving

                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                  ddU IUU t

                                                                  yxdyxyxyxUyUxUdn

                                                                  i ii 2

                                                                  1

                                                                  U

                                                                  Review of Linear Algebra (Cont)

                                                                  Singular Value Decomposition (SVD)

                                                                  For a Matrix

                                                                  Find

                                                                  ndX

                                                                  Review of Linear Algebra (Cont)

                                                                  Singular Value Decomposition (SVD)

                                                                  For a Matrix

                                                                  Find a Diagonal Matrix

                                                                  with Entries

                                                                  called Singular Values

                                                                  ndX

                                                                  ndS

                                                                  )min(1 ndss

                                                                  Review of Linear Algebra (Cont)

                                                                  Singular Value Decomposition (SVD)

                                                                  For a Matrix

                                                                  Find a Diagonal Matrix

                                                                  with Entries

                                                                  called Singular Values

                                                                  And Unitary (Rotation) Matrices

                                                                  (recall )

                                                                  ndX

                                                                  ndS

                                                                  )min(1 ndss

                                                                  ddU nnV

                                                                  IVVUU tt

                                                                  Review of Linear Algebra (Cont)

                                                                  Singular Value Decomposition (SVD)

                                                                  For a Matrix

                                                                  Find a Diagonal Matrix

                                                                  with Entries

                                                                  called Singular Values

                                                                  And Unitary (Rotation) Matrices

                                                                  (recall )

                                                                  So That

                                                                  ndX

                                                                  ndS

                                                                  )min(1 ndss

                                                                  ddU nnV

                                                                  IVVUU tt tUSVX

                                                                  Review of Linear Algebra (Cont)

                                                                  Intuition behind Singular Value Decomposition

                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                  vVSUvVSUvX tt

                                                                  Review of Linear Algebra (Cont)

                                                                  Intuition behind Singular Value Decomposition

                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                  bull First rotate

                                                                  X

                                                                  vVSUvVSUvX tt

                                                                  Review of Linear Algebra (Cont)

                                                                  Intuition behind Singular Value Decomposition

                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                  bull First rotate

                                                                  bull Second rescale coordinate axes (by )

                                                                  X

                                                                  vVSUvVSUvX tt

                                                                  is

                                                                  Review of Linear Algebra (Cont)

                                                                  Intuition behind Singular Value Decomposition

                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                  bull First rotate

                                                                  bull Second rescale coordinate axes (by )

                                                                  bull Third rotate again

                                                                  X

                                                                  vVSUvVSUvX tt

                                                                  is

                                                                  Review of Linear Algebra (Cont)

                                                                  Intuition behind Singular Value Decomposition

                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                  bull First rotate

                                                                  bull Second rescale coordinate axes (by )

                                                                  bull Third rotate again

                                                                  bull ie have diagonalized the transformation

                                                                  X

                                                                  vVSUvVSUvX tt

                                                                  is

                                                                  SVD Compact Representation

                                                                  Useful Labeling

                                                                  Singular Values in Increasing Order

                                                                  Review of Linear Algebra (Cont)

                                                                  )min(1 dnss

                                                                  SVD Compact Representation

                                                                  Useful Labeling

                                                                  Singular Values in Increasing Order

                                                                  Note singular values = 0 can be omitted

                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                  Review of Linear Algebra (Cont)

                                                                  )min(1 dnss

                                                                  Review of Linear Algebra (Cont)

                                                                  r

                                                                  SVD Compact Representation

                                                                  Useful Labeling

                                                                  Singular Values in Increasing Order

                                                                  Note singular values = 0 can be omitted

                                                                  Let = of positive singular values

                                                                  )min(1 dnss

                                                                  Review of Linear Algebra (Cont)

                                                                  r

                                                                  SVD Compact Representation

                                                                  Useful Labeling

                                                                  Singular Values in Increasing Order

                                                                  Note singular values = 0 can be omitted

                                                                  Let = of positive singular values

                                                                  Then

                                                                  Where are truncations of

                                                                  trnrrrd VSUX

                                                                  VSU

                                                                  )min(1 dnss

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Full Representation

                                                                  =

                                                                  Graphics Display Assumes

                                                                  ndX ddU ndS nn

                                                                  tV

                                                                  nd

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Full Representation

                                                                  =

                                                                  Full Rank Basis Matrix

                                                                  ndX ddU ndS nn

                                                                  tV

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Full Representation

                                                                  =

                                                                  Full Rank Basis Matrix

                                                                  All 0s in Bottom

                                                                  ndX ddU ndS nn

                                                                  tV

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Reduced Representation

                                                                  =

                                                                  These Columns Get 0ed Out

                                                                  ndX ddU nnS nn

                                                                  tV

                                                                  nnd 0

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Reduced Representation

                                                                  =ndX ndU nnS nn

                                                                  tV

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Reduced Representation

                                                                  =

                                                                  Also Some of These May be 0

                                                                  ndX ndU nnS nn

                                                                  tV

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Compact Representation

                                                                  = ndX rdU

                                                                  rrS nrtV

                                                                  0

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Compact Representation

                                                                  =

                                                                  These Get 0ed Out

                                                                  ndX rdU

                                                                  rrS nrtV

                                                                  0

                                                                  Review of Linear Algebra (Cont)

                                                                  SVD Compact Representation

                                                                  = ndX rdU

                                                                  rrS nrtV

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition

                                                                  For a (Symmetric) Square Matrix

                                                                  Find

                                                                  ddX

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition

                                                                  For a (Symmetric) Square Matrix

                                                                  Find a Diagonal Matrix

                                                                  Called Eigenvalues

                                                                  Convenient Ordering

                                                                  ddX

                                                                  d

                                                                  D

                                                                  0

                                                                  01

                                                                  n 1

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition

                                                                  For a (Symmetric) Square Matrix

                                                                  Find a Diagonal Matrix

                                                                  And an Orthonormal Matrix

                                                                  (ie )

                                                                  ddX

                                                                  d

                                                                  D

                                                                  0

                                                                  01

                                                                  ddB

                                                                  ddtt IBBBB

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition

                                                                  For a (Symmetric) Square Matrix

                                                                  Find a Diagonal Matrix

                                                                  And an Orthonormal Matrix

                                                                  (ie )

                                                                  So that ie

                                                                  ddX

                                                                  d

                                                                  D

                                                                  0

                                                                  01

                                                                  ddB

                                                                  ddtt IBBBB

                                                                  DBBX tBDBX

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                  (looks similar)

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                  Complex (uses )

                                                                  VU

                                                                  1i

                                                                  Review of Linear Algebra (Cont)

                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                  VU

                                                                  X

                                                                  BVU

                                                                  1i

                                                                  • Participant Presentations
                                                                  • PCA to find clusters
                                                                  • Statistical Smoothing
                                                                  • Kernel Density Estimation
                                                                  • Scatterplot Smoothing
                                                                  • Statistical Smoothing (2)
                                                                  • SiZer Background
                                                                  • SiZer Background (2)
                                                                  • SiZer Background (3)
                                                                  • SiZer Background (4)
                                                                  • SiZer Background (5)
                                                                  • SiZer Overview
                                                                  • PCA to find clusters (2)
                                                                  • PCA to find clusters (3)
                                                                  • PCA to find clusters (4)
                                                                  • PCA to find clusters (5)
                                                                  • PCA to find clusters (6)
                                                                  • PCA to find clusters (7)
                                                                  • Recall Yeast Cell Cycle Data
                                                                  • Yeast Cell Cycle Data FDA View
                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                  • Frequency 2 Analysis
                                                                  • Frequency 2 Analysis (2)
                                                                  • Frequency 2 Analysis (3)
                                                                  • Yeast Cell Cycle
                                                                  • SiZer Study of Distrsquon of Angles
                                                                  • Reclassification of Major Genes
                                                                  • Compare to Previous Classifrsquon
                                                                  • New Subpopulation View
                                                                  • New Subpopulation View (2)
                                                                  • Detailed Look at PCA
                                                                  • PCA Rediscovery ndash Renaming
                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                  • An Interesting Historical Note
                                                                  • An Interesting Historical Note (2)
                                                                  • Detailed Look at PCA (2)
                                                                  • Detailed Look at PCA (3)
                                                                  • Review of Linear Algebra
                                                                  • Review of Linear Algebra (2)
                                                                  • Review of Linear Algebra (3)
                                                                  • Review of Linear Algebra (Cont)
                                                                  • Review of Linear Algebra (Cont) (2)
                                                                  • Review of Linear Algebra (Cont) (3)
                                                                  • Review of Linear Algebra (Cont) (4)
                                                                  • Review of Linear Algebra (Cont) (5)
                                                                  • Review of Linear Algebra (Cont) (6)
                                                                  • Review of Linear Algebra (Cont) (7)
                                                                  • Review of Linear Algebra (Cont) (8)
                                                                  • Review of Linear Algebra (Cont) (9)
                                                                  • Review of Linear Algebra (Cont) (10)
                                                                  • Review of Linear Algebra (Cont) (11)
                                                                  • Review of Linear Algebra (Cont) (12)
                                                                  • Review of Linear Algebra (Cont) (13)
                                                                  • Review of Linear Algebra (Cont) (14)
                                                                  • Review of Linear Algebra (Cont) (15)
                                                                  • Review of Linear Algebra (Cont) (16)
                                                                  • Review of Linear Algebra (Cont) (17)
                                                                  • Review of Linear Algebra (Cont) (18)
                                                                  • Review of Linear Algebra (Cont) (19)
                                                                  • Review of Linear Algebra (Cont) (20)
                                                                  • Review of Linear Algebra (Cont) (21)
                                                                  • Review of Linear Algebra (Cont) (22)
                                                                  • Review of Linear Algebra (Cont) (23)
                                                                  • Review of Linear Algebra (Cont) (24)
                                                                  • Review of Linear Algebra (Cont) (25)
                                                                  • Review of Linear Algebra (Cont) (26)
                                                                  • Review of Linear Algebra (Cont) (27)
                                                                  • Review of Linear Algebra (Cont) (28)
                                                                  • Review of Linear Algebra (Cont) (29)
                                                                  • Review of Linear Algebra (Cont) (30)
                                                                  • Review of Linear Algebra (Cont) (31)
                                                                  • Review of Linear Algebra (Cont) (32)
                                                                  • Review of Linear Algebra (Cont) (33)
                                                                  • Review of Linear Algebra (Cont) (34)
                                                                  • Review of Linear Algebra (Cont) (35)
                                                                  • Review of Linear Algebra (Cont) (36)
                                                                  • Review of Linear Algebra (Cont) (37)
                                                                  • Review of Linear Algebra (Cont) (38)
                                                                  • Review of Linear Algebra (Cont) (39)
                                                                  • Review of Linear Algebra (Cont) (40)
                                                                  • Review of Linear Algebra (Cont) (41)
                                                                  • Review of Linear Algebra (Cont) (42)
                                                                  • Review of Linear Algebra (Cont) (43)
                                                                  • Review of Linear Algebra (Cont) (44)
                                                                  • Review of Linear Algebra (Cont) (45)
                                                                  • Review of Linear Algebra (Cont) (46)
                                                                  • Review of Linear Algebra (Cont) (47)
                                                                  • Review of Linear Algebra (Cont) (48)
                                                                  • Review of Linear Algebra (Cont) (49)
                                                                  • Review of Linear Algebra (Cont) (50)
                                                                  • Review of Linear Algebra (Cont) (51)
                                                                  • Review of Linear Algebra (Cont) (52)
                                                                  • Review of Linear Algebra (Cont) (53)
                                                                  • Review of Linear Algebra (Cont) (54)
                                                                  • Review of Linear Algebra (Cont) (55)
                                                                  • Review of Linear Algebra (Cont) (56)
                                                                  • Review of Linear Algebra (Cont) (57)
                                                                  • Review of Linear Algebra (Cont) (58)
                                                                  • Review of Linear Algebra (Cont) (59)
                                                                  • Review of Linear Algebra (Cont) (60)
                                                                  • Review of Linear Algebra (Cont) (61)
                                                                  • Review of Linear Algebra (Cont) (62)
                                                                  • Review of Linear Algebra (Cont) (63)
                                                                  • Review of Linear Algebra (Cont) (64)
                                                                  • Review of Linear Algebra (Cont) (65)
                                                                  • Review of Linear Algebra (Cont) (66)
                                                                  • Review of Linear Algebra (Cont) (67)
                                                                  • Review of Linear Algebra (Cont) (68)
                                                                  • Review of Linear Algebra (Cont) (69)
                                                                  • Review of Linear Algebra (Cont) (70)
                                                                  • Review of Linear Algebra (Cont) (71)
                                                                  • Review of Linear Algebra (Cont) (72)
                                                                  • Review of Linear Algebra (Cont) (73)
                                                                  • Review of Linear Algebra (Cont) (74)
                                                                  • Review of Linear Algebra (Cont) (75)
                                                                  • Review of Linear Algebra (Cont) (76)
                                                                  • Review of Linear Algebra (Cont) (77)
                                                                  • Review of Linear Algebra (Cont) (78)
                                                                  • Review of Linear Algebra (Cont) (79)
                                                                  • Review of Linear Algebra (Cont) (80)
                                                                  • Review of Linear Algebra (Cont) (81)

                                                                    PCA Rediscovery ndash Renaming

                                                                    Statistics Principal Component Analysis (PCA)

                                                                    PCA Rediscovery ndash Renaming

                                                                    Statistics Principal Component Analysis (PCA)

                                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                                    PCA Rediscovery ndash Renaming

                                                                    Statistics Principal Component Analysis (PCA)

                                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                                    PCA Rediscovery ndash Renaming

                                                                    Statistics Principal Component Analysis (PCA)

                                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                    PCA Rediscovery ndash Renaming

                                                                    Statistics Principal Component Analysis (PCA)

                                                                    Social Sciences Factor Analysis (PCA is a subset)

                                                                    Probability Electrical EngKarhunen ndash Loeve expansion

                                                                    Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                    Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                    An Interesting Historical Note

                                                                    The 1st () application of PCA to Functional

                                                                    Data Analysis

                                                                    An Interesting Historical Note

                                                                    The 1st () application of PCA to Functional

                                                                    Data Analysis

                                                                    Rao (1958)

                                                                    1st Paper with ldquoCurves as Data Objectsrdquo

                                                                    viewpoint

                                                                    Detailed Look at PCA

                                                                    Three Important (amp Interesting) Viewpoints

                                                                    1 Mathematics

                                                                    2 Numerics

                                                                    3 Statistics

                                                                    Goal Study Interrelationships

                                                                    Detailed Look at PCA

                                                                    Three Important (amp Interesting) Viewpoints

                                                                    1 Mathematics

                                                                    2 Numerics

                                                                    3 Statistics

                                                                    1st Review Linear Alg and Multivar Prob

                                                                    Review of Linear Algebra

                                                                    Vector Space

                                                                    bull set of ldquovectorsrdquo

                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                    xa

                                                                    Review of Linear Algebra

                                                                    Vector Space

                                                                    bull set of ldquovectorsrdquo

                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                    ( in space)

                                                                    xa

                                                                    i

                                                                    ii xa

                                                                    Review of Linear Algebra

                                                                    Vector Space

                                                                    bull set of ldquovectorsrdquo

                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                    ( in space)

                                                                    eg

                                                                    ldquo dim Euclidrsquon spacerdquo

                                                                    xa

                                                                    i

                                                                    ii xa

                                                                    d

                                                                    d

                                                                    d xx

                                                                    x

                                                                    x

                                                                    x 1

                                                                    1

                                                                    d

                                                                    Review of Linear Algebra (Cont)

                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                    Review of Linear Algebra (Cont)

                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                    Review of Linear Algebra (Cont)

                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                    Note Planes not Through the Origin

                                                                    are not Subspaces

                                                                    (Do not Contain )00 x

                                                                    Review of Linear Algebra (Cont)

                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                    (all Linear Combos of them =

                                                                    = Containing Hyperplane

                                                                    through Origin)

                                                                    Review of Linear Algebra (Cont)

                                                                    Basis of Subspace Set of Vectors that

                                                                    bull Span ie Everything is a Lin Com of them

                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                    Review of Linear Algebra (Cont)

                                                                    Basis of Subspace Set of Vectors that

                                                                    bull Span ie Everything is a Lin Com of them

                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                    bull eg ldquoUnit Vector Basisrdquo

                                                                    bull Since

                                                                    d

                                                                    1

                                                                    0

                                                                    0

                                                                    0

                                                                    1

                                                                    0

                                                                    0

                                                                    0

                                                                    1

                                                                    1

                                                                    0

                                                                    0

                                                                    0

                                                                    1

                                                                    0

                                                                    0

                                                                    0

                                                                    1

                                                                    212

                                                                    1

                                                                    d

                                                                    d

                                                                    xxx

                                                                    x

                                                                    x

                                                                    x

                                                                    Review of Linear Algebra (Cont)

                                                                    Basis Matrix of subspace of

                                                                    Given a basis

                                                                    create matrix of columns

                                                                    dnvv 1

                                                                    nddnd

                                                                    n

                                                                    n

                                                                    vv

                                                                    vv

                                                                    vvB

                                                                    1

                                                                    111

                                                                    1

                                                                    Review of Linear Algebra (Cont)

                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                    where

                                                                    n

                                                                    iii aBva

                                                                    1

                                                                    na

                                                                    a

                                                                    a 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                    where

                                                                    Note Right Multiplication Gives

                                                                    Linear Combination of Column Vectors

                                                                    n

                                                                    iii aBva

                                                                    1

                                                                    na

                                                                    a

                                                                    a 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                    where

                                                                    Check sizes

                                                                    n

                                                                    iii aBva

                                                                    1

                                                                    na

                                                                    a

                                                                    a 1

                                                                    )1()(1 nndd

                                                                    Review of Linear Algebra (Cont)

                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                    For matrices

                                                                    Define the Matrix Product

                                                                    mkk

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    nmm

                                                                    n

                                                                    bb

                                                                    bb

                                                                    B

                                                                    1

                                                                    111

                                                                    m

                                                                    iniik

                                                                    m

                                                                    iiik

                                                                    m

                                                                    inii

                                                                    m

                                                                    iii

                                                                    baba

                                                                    baba

                                                                    AB

                                                                    1

                                                                    11

                                                                    11

                                                                    111

                                                                    Review of Linear Algebra (Cont)

                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                    For matrices

                                                                    Define the Matrix Product

                                                                    (Inner Products of Rows With Columns )

                                                                    mkk

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    nmm

                                                                    n

                                                                    bb

                                                                    bb

                                                                    B

                                                                    1

                                                                    111

                                                                    m

                                                                    iniik

                                                                    m

                                                                    iiik

                                                                    m

                                                                    inii

                                                                    m

                                                                    iii

                                                                    baba

                                                                    baba

                                                                    AB

                                                                    1

                                                                    11

                                                                    11

                                                                    111

                                                                    A B

                                                                    Review of Linear Algebra (Cont)

                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                    For matrices

                                                                    Define the Matrix Product

                                                                    (Inner Products of Rows With Columns )

                                                                    (Composition of Linear Transformations)

                                                                    mkk

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    nmm

                                                                    n

                                                                    bb

                                                                    bb

                                                                    B

                                                                    1

                                                                    111

                                                                    m

                                                                    iniik

                                                                    m

                                                                    iiik

                                                                    m

                                                                    inii

                                                                    m

                                                                    iii

                                                                    baba

                                                                    baba

                                                                    AB

                                                                    1

                                                                    11

                                                                    11

                                                                    111

                                                                    A B

                                                                    Review of Linear Algebra (Cont)

                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                    For matrices

                                                                    Define the Matrix Product

                                                                    (Inner Products of Rows With Columns )

                                                                    (Composition of Linear Transformations)

                                                                    Often Useful to Check Sizes

                                                                    mkk

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    nmm

                                                                    n

                                                                    bb

                                                                    bb

                                                                    B

                                                                    1

                                                                    111

                                                                    m

                                                                    iniik

                                                                    m

                                                                    iiik

                                                                    m

                                                                    inii

                                                                    m

                                                                    iii

                                                                    baba

                                                                    baba

                                                                    AB

                                                                    1

                                                                    11

                                                                    11

                                                                    111

                                                                    nmmknk

                                                                    A B

                                                                    Review of Linear Algebra (Cont)

                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                    For matrices

                                                                    Define the Matrix Product

                                                                    (Inner Products of Rows With Columns )

                                                                    (Composition of Linear Transformations)

                                                                    Often Useful to Check Sizes

                                                                    mkk

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    nmm

                                                                    n

                                                                    bb

                                                                    bb

                                                                    B

                                                                    1

                                                                    111

                                                                    m

                                                                    iniik

                                                                    m

                                                                    iiik

                                                                    m

                                                                    inii

                                                                    m

                                                                    iii

                                                                    baba

                                                                    baba

                                                                    AB

                                                                    1

                                                                    11

                                                                    11

                                                                    111

                                                                    nmmknk

                                                                    A B

                                                                    Review of Linear Algebra (Cont)

                                                                    Matrix Trace

                                                                    bull For a Square Matrix

                                                                    bull Define

                                                                    mmm

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    m

                                                                    iiiaAtr

                                                                    1)(

                                                                    Review of Linear Algebra (Cont)

                                                                    Matrix Trace

                                                                    bull For a Square Matrix

                                                                    bull Define

                                                                    bull Trace Commutes with Matrix Multiplication

                                                                    mmm

                                                                    m

                                                                    aa

                                                                    aa

                                                                    A

                                                                    1

                                                                    111

                                                                    m

                                                                    iiiaAtr

                                                                    1)(

                                                                    BAtrABtr

                                                                    Review of Linear Algebra (Cont)

                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                    bull Number of Elements in a Basis (Unique)

                                                                    Review of Linear Algebra (Cont)

                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                    bull Number of Elements in a Basis (Unique)

                                                                    bull (Use Basis Above)

                                                                    bull eg dim of a line is 1

                                                                    bull eg dim of a plane is 2

                                                                    dd dim

                                                                    Review of Linear Algebra (Cont)

                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                    bull Number of Elements in a Basis (Unique)

                                                                    bull (Use Basis Above)

                                                                    bull eg dim of a line is 1

                                                                    bull eg dim of a plane is 2

                                                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                                                    (in Statistical Uses eg ANOVA)

                                                                    dd dim

                                                                    Review of Linear Algebra (Cont)

                                                                    Norm of a Vector

                                                                    bull in d 21

                                                                    21

                                                                    1

                                                                    2 xxxx td

                                                                    jj

                                                                    Review of Linear Algebra (Cont)

                                                                    Norm of a Vector

                                                                    bull in

                                                                    bull Idea length of the vector

                                                                    d 2121

                                                                    1

                                                                    2 xxxx td

                                                                    jj

                                                                    Review of Linear Algebra (Cont)

                                                                    Norm of a Vector

                                                                    bull in

                                                                    bull Idea length of the vector

                                                                    bull Note strange properties for high

                                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                                    d 2121

                                                                    1

                                                                    2 xxxx td

                                                                    jj

                                                                    d

                                                                    d

                                                                    Review of Linear Algebra (Cont)

                                                                    Norm of a Vector (cont)

                                                                    bull Length Normalized Vector

                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                    amp is a Direction Vector)

                                                                    x

                                                                    x

                                                                    Review of Linear Algebra (Cont)

                                                                    Norm of a Vector (cont)

                                                                    bull Length Normalized Vector

                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                    amp is a Direction Vector)

                                                                    bull Define Distance as

                                                                    x

                                                                    x

                                                                    yxyxyxyxd t

                                                                    Review of Linear Algebra (Cont)

                                                                    Inner (Dot Scalar) Product

                                                                    bull for Vectors and

                                                                    yxyxyx td

                                                                    jjj

                                                                    1

                                                                    x y

                                                                    Review of Linear Algebra (Cont)

                                                                    Inner (Dot Scalar) Product

                                                                    bull for Vectors and

                                                                    bull Related to Norm via

                                                                    yxyxyx td

                                                                    jjj

                                                                    1

                                                                    x y

                                                                    21 xxx

                                                                    Review of Linear Algebra (Cont)

                                                                    Inner (Dot Scalar) Product (cont)

                                                                    bull measures ldquoangle between and rdquo as

                                                                    yyxx

                                                                    yx

                                                                    yx

                                                                    yxyxangle

                                                                    tt

                                                                    t

                                                                    11 cos

                                                                    cos

                                                                    x y

                                                                    Review of Linear Algebra (Cont)

                                                                    Inner (Dot Scalar) Product (cont)

                                                                    bull measures ldquoangle between and rdquo as

                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                    if and only if

                                                                    yyxx

                                                                    yx

                                                                    yx

                                                                    yxyxangle

                                                                    tt

                                                                    t

                                                                    11 cos

                                                                    cos

                                                                    x y

                                                                    yx 0 yx

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis

                                                                    bull All Orthogonal to each other

                                                                    ie for

                                                                    bull All have Length 1

                                                                    ie for

                                                                    nvv 1

                                                                    1 ii vv

                                                                    0 ii vv ii

                                                                    ni 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis (cont)

                                                                    bull Spectral Representation

                                                                    where

                                                                    nvv 1

                                                                    n

                                                                    iii vax

                                                                    1

                                                                    ii vxa

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis (cont)

                                                                    bull Spectral Representation

                                                                    where

                                                                    (Coefficient is Inner Product Cool Notation)

                                                                    nvv 1

                                                                    n

                                                                    iii vax

                                                                    1

                                                                    ii vxa

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis (cont)

                                                                    bull Spectral Representation

                                                                    where

                                                                    Check

                                                                    nvv 1

                                                                    n

                                                                    iii vax

                                                                    1

                                                                    ii vxa

                                                                    iii

                                                                    n

                                                                    iii

                                                                    n

                                                                    iiii avvavvavx

                                                                    1

                                                                    1

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis (cont)

                                                                    bull Spectral Representation

                                                                    where

                                                                    Check

                                                                    bull Matrix Notation where ie

                                                                    For the Basis Matrix

                                                                    nvv 1

                                                                    n

                                                                    iii vax

                                                                    1

                                                                    ii vxa

                                                                    iii

                                                                    n

                                                                    iii

                                                                    n

                                                                    iiii avvavvavx

                                                                    1

                                                                    1

                                                                    aBx Bxa tt xBa t

                                                                    nvvB 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Orthonormal Basis (cont)

                                                                    bull Spectral Representation

                                                                    where

                                                                    Check

                                                                    bull Matrix Notation where ie

                                                                    is called transform of

                                                                    (eg Fourier or Wavelet)

                                                                    nvv 1

                                                                    n

                                                                    iii vax

                                                                    1

                                                                    ii vxa

                                                                    iii

                                                                    n

                                                                    iii

                                                                    n

                                                                    iiii avvavvavx

                                                                    1

                                                                    1

                                                                    aBx Bxa tt xBa t

                                                                    xa

                                                                    Review of Linear Algebra (Cont)

                                                                    Parseval identity for

                                                                    in subsp genrsquod by o n basis

                                                                    x

                                                                    nvv 1

                                                                    2

                                                                    1

                                                                    22

                                                                    1

                                                                    2 aavxx

                                                                    n

                                                                    ii

                                                                    n

                                                                    ii

                                                                    Review of Linear Algebra (Cont)

                                                                    Parseval identity for

                                                                    in subsp genrsquod by o n basis

                                                                    bull Pythagorean theorem

                                                                    bull ldquoDecomposition of Energyrdquo

                                                                    bull ANOVA - sums of squares

                                                                    x

                                                                    nvv 1

                                                                    2

                                                                    1

                                                                    22

                                                                    1

                                                                    2 aavxx

                                                                    n

                                                                    ii

                                                                    n

                                                                    ii

                                                                    Review of Linear Algebra (Cont)

                                                                    Parseval identity for

                                                                    in subsp genrsquod by o n basis

                                                                    bull Pythagorean theorem

                                                                    bull ldquoDecomposition of Energyrdquo

                                                                    bull ANOVA - sums of squares

                                                                    bull Transform has same length as

                                                                    ie ldquorotation in rdquo

                                                                    x

                                                                    nvv 1

                                                                    2

                                                                    1

                                                                    22

                                                                    1

                                                                    2 aavxx

                                                                    n

                                                                    ii

                                                                    n

                                                                    ii

                                                                    a xd

                                                                    Projection of a Vector onto a Subspace

                                                                    bull Idea Member of that is Closest to

                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                    Review of Linear Algebra (Cont)x

                                                                    xV

                                                                    V

                                                                    Projection of a Vector onto a Subspace

                                                                    bull Idea Member of that is Closest to

                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                    bull Find that Solves

                                                                    (ldquoLeast Squaresrdquo)

                                                                    Review of Linear Algebra (Cont)x

                                                                    xV

                                                                    V

                                                                    VxPV vxVv

                                                                    min

                                                                    Projection of a Vector onto a Subspace

                                                                    bull Idea Member of that is Closest to

                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                    bull Find that Solves

                                                                    (ldquoLeast Squaresrdquo)

                                                                    bull For Inner Product (Hilbert) Space

                                                                    Exists and is Unique

                                                                    Review of Linear Algebra (Cont)x

                                                                    xV

                                                                    V

                                                                    VxPV vxVv

                                                                    min

                                                                    xPV

                                                                    Projection of a Vector onto a Subspace (cont)

                                                                    bull General Solution in for Basis Matrix

                                                                    Review of Linear Algebra (Cont)

                                                                    d VB

                                                                    xBBBBxP tVV

                                                                    tVVV

                                                                    1

                                                                    Projection of a Vector onto a Subspace (cont)

                                                                    bull General Solution in for Basis Matrix

                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                    (thus projection is another linear operation)

                                                                    Review of Linear Algebra (Cont)

                                                                    d VB

                                                                    xBBBBxP tVV

                                                                    tVVV

                                                                    1

                                                                    tVV

                                                                    tVVV BBBBP

                                                                    1

                                                                    Projection of a Vector onto a Subspace (cont)

                                                                    bull General Solution in for Basis Matrix

                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                    (thus projection is another linear operation)

                                                                    (note same operation underlies least squares)

                                                                    Review of Linear Algebra (Cont)

                                                                    d VB

                                                                    xBBBBxP tVV

                                                                    tVVV

                                                                    1

                                                                    tVV

                                                                    tVVV BBBBP

                                                                    1

                                                                    Review of Linear Algebra (Cont)

                                                                    Projection using Orthonormal Basis

                                                                    bull Basis Matrix is Orthonormal

                                                                    nnVtV IBB

                                                                    10

                                                                    01

                                                                    1

                                                                    111

                                                                    1

                                                                    1

                                                                    nnn

                                                                    n

                                                                    ntn

                                                                    t

                                                                    vvvv

                                                                    vvvv

                                                                    vv

                                                                    v

                                                                    v

                                                                    nvv 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Projection using Orthonormal Basis

                                                                    bull Basis Matrix is Orthonormal

                                                                    bull So =

                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                    (Recall Right Multrsquon)

                                                                    nnVtV IBB

                                                                    10

                                                                    01

                                                                    1

                                                                    111

                                                                    1

                                                                    1

                                                                    nnn

                                                                    n

                                                                    ntn

                                                                    t

                                                                    vvvv

                                                                    vvvv

                                                                    vv

                                                                    v

                                                                    v

                                                                    xBBxP tVVV

                                                                    x V

                                                                    nvv 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Projection using Orthonormal Basis (cont)

                                                                    bull For Orthogonal Complement

                                                                    and

                                                                    V

                                                                    xPxPx VV 222xPxPx VV

                                                                    Review of Linear Algebra (Cont)

                                                                    Projection using Orthonormal Basis (cont)

                                                                    bull For Orthogonal Complement

                                                                    and

                                                                    bull Parseval Inequality

                                                                    V

                                                                    xPxPx VV 222xPxPx VV

                                                                    2

                                                                    1

                                                                    22

                                                                    1

                                                                    22 aavxxxP

                                                                    n

                                                                    ii

                                                                    n

                                                                    iiV

                                                                    Review of Linear Algebra (Cont)

                                                                    (Real) Unitary Matrices with

                                                                    bull Orthonormal Basis Matrix

                                                                    (So All of Above Applies)

                                                                    ddU IUU t

                                                                    Review of Linear Algebra (Cont)

                                                                    (Real) Unitary Matrices with

                                                                    bull Orthonormal Basis Matrix

                                                                    (So All of Above Applies)

                                                                    bull Note Transformrsquon is Distance Preserving

                                                                    ddU IUU t

                                                                    yxdyxyxyxUyUxUdn

                                                                    i ii 2

                                                                    1

                                                                    Review of Linear Algebra (Cont)

                                                                    (Real) Unitary Matrices with

                                                                    bull Orthonormal Basis Matrix

                                                                    (So All of Above Applies)

                                                                    bull Note Transformrsquon is Distance Preserving

                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                    ddU IUU t

                                                                    yxdyxyxyxUyUxUdn

                                                                    i ii 2

                                                                    1

                                                                    U

                                                                    Review of Linear Algebra (Cont)

                                                                    Singular Value Decomposition (SVD)

                                                                    For a Matrix

                                                                    Find

                                                                    ndX

                                                                    Review of Linear Algebra (Cont)

                                                                    Singular Value Decomposition (SVD)

                                                                    For a Matrix

                                                                    Find a Diagonal Matrix

                                                                    with Entries

                                                                    called Singular Values

                                                                    ndX

                                                                    ndS

                                                                    )min(1 ndss

                                                                    Review of Linear Algebra (Cont)

                                                                    Singular Value Decomposition (SVD)

                                                                    For a Matrix

                                                                    Find a Diagonal Matrix

                                                                    with Entries

                                                                    called Singular Values

                                                                    And Unitary (Rotation) Matrices

                                                                    (recall )

                                                                    ndX

                                                                    ndS

                                                                    )min(1 ndss

                                                                    ddU nnV

                                                                    IVVUU tt

                                                                    Review of Linear Algebra (Cont)

                                                                    Singular Value Decomposition (SVD)

                                                                    For a Matrix

                                                                    Find a Diagonal Matrix

                                                                    with Entries

                                                                    called Singular Values

                                                                    And Unitary (Rotation) Matrices

                                                                    (recall )

                                                                    So That

                                                                    ndX

                                                                    ndS

                                                                    )min(1 ndss

                                                                    ddU nnV

                                                                    IVVUU tt tUSVX

                                                                    Review of Linear Algebra (Cont)

                                                                    Intuition behind Singular Value Decomposition

                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                    vVSUvVSUvX tt

                                                                    Review of Linear Algebra (Cont)

                                                                    Intuition behind Singular Value Decomposition

                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                    bull First rotate

                                                                    X

                                                                    vVSUvVSUvX tt

                                                                    Review of Linear Algebra (Cont)

                                                                    Intuition behind Singular Value Decomposition

                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                    bull First rotate

                                                                    bull Second rescale coordinate axes (by )

                                                                    X

                                                                    vVSUvVSUvX tt

                                                                    is

                                                                    Review of Linear Algebra (Cont)

                                                                    Intuition behind Singular Value Decomposition

                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                    bull First rotate

                                                                    bull Second rescale coordinate axes (by )

                                                                    bull Third rotate again

                                                                    X

                                                                    vVSUvVSUvX tt

                                                                    is

                                                                    Review of Linear Algebra (Cont)

                                                                    Intuition behind Singular Value Decomposition

                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                    bull First rotate

                                                                    bull Second rescale coordinate axes (by )

                                                                    bull Third rotate again

                                                                    bull ie have diagonalized the transformation

                                                                    X

                                                                    vVSUvVSUvX tt

                                                                    is

                                                                    SVD Compact Representation

                                                                    Useful Labeling

                                                                    Singular Values in Increasing Order

                                                                    Review of Linear Algebra (Cont)

                                                                    )min(1 dnss

                                                                    SVD Compact Representation

                                                                    Useful Labeling

                                                                    Singular Values in Increasing Order

                                                                    Note singular values = 0 can be omitted

                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                    Review of Linear Algebra (Cont)

                                                                    )min(1 dnss

                                                                    Review of Linear Algebra (Cont)

                                                                    r

                                                                    SVD Compact Representation

                                                                    Useful Labeling

                                                                    Singular Values in Increasing Order

                                                                    Note singular values = 0 can be omitted

                                                                    Let = of positive singular values

                                                                    )min(1 dnss

                                                                    Review of Linear Algebra (Cont)

                                                                    r

                                                                    SVD Compact Representation

                                                                    Useful Labeling

                                                                    Singular Values in Increasing Order

                                                                    Note singular values = 0 can be omitted

                                                                    Let = of positive singular values

                                                                    Then

                                                                    Where are truncations of

                                                                    trnrrrd VSUX

                                                                    VSU

                                                                    )min(1 dnss

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Full Representation

                                                                    =

                                                                    Graphics Display Assumes

                                                                    ndX ddU ndS nn

                                                                    tV

                                                                    nd

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Full Representation

                                                                    =

                                                                    Full Rank Basis Matrix

                                                                    ndX ddU ndS nn

                                                                    tV

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Full Representation

                                                                    =

                                                                    Full Rank Basis Matrix

                                                                    All 0s in Bottom

                                                                    ndX ddU ndS nn

                                                                    tV

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Reduced Representation

                                                                    =

                                                                    These Columns Get 0ed Out

                                                                    ndX ddU nnS nn

                                                                    tV

                                                                    nnd 0

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Reduced Representation

                                                                    =ndX ndU nnS nn

                                                                    tV

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Reduced Representation

                                                                    =

                                                                    Also Some of These May be 0

                                                                    ndX ndU nnS nn

                                                                    tV

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Compact Representation

                                                                    = ndX rdU

                                                                    rrS nrtV

                                                                    0

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Compact Representation

                                                                    =

                                                                    These Get 0ed Out

                                                                    ndX rdU

                                                                    rrS nrtV

                                                                    0

                                                                    Review of Linear Algebra (Cont)

                                                                    SVD Compact Representation

                                                                    = ndX rdU

                                                                    rrS nrtV

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition

                                                                    For a (Symmetric) Square Matrix

                                                                    Find

                                                                    ddX

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition

                                                                    For a (Symmetric) Square Matrix

                                                                    Find a Diagonal Matrix

                                                                    Called Eigenvalues

                                                                    Convenient Ordering

                                                                    ddX

                                                                    d

                                                                    D

                                                                    0

                                                                    01

                                                                    n 1

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition

                                                                    For a (Symmetric) Square Matrix

                                                                    Find a Diagonal Matrix

                                                                    And an Orthonormal Matrix

                                                                    (ie )

                                                                    ddX

                                                                    d

                                                                    D

                                                                    0

                                                                    01

                                                                    ddB

                                                                    ddtt IBBBB

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition

                                                                    For a (Symmetric) Square Matrix

                                                                    Find a Diagonal Matrix

                                                                    And an Orthonormal Matrix

                                                                    (ie )

                                                                    So that ie

                                                                    ddX

                                                                    d

                                                                    D

                                                                    0

                                                                    01

                                                                    ddB

                                                                    ddtt IBBBB

                                                                    DBBX tBDBX

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                    (looks similar)

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                    Complex (uses )

                                                                    VU

                                                                    1i

                                                                    Review of Linear Algebra (Cont)

                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                    VU

                                                                    X

                                                                    BVU

                                                                    1i

                                                                    • Participant Presentations
                                                                    • PCA to find clusters
                                                                    • Statistical Smoothing
                                                                    • Kernel Density Estimation
                                                                    • Scatterplot Smoothing
                                                                    • Statistical Smoothing (2)
                                                                    • SiZer Background
                                                                    • SiZer Background (2)
                                                                    • SiZer Background (3)
                                                                    • SiZer Background (4)
                                                                    • SiZer Background (5)
                                                                    • SiZer Overview
                                                                    • PCA to find clusters (2)
                                                                    • PCA to find clusters (3)
                                                                    • PCA to find clusters (4)
                                                                    • PCA to find clusters (5)
                                                                    • PCA to find clusters (6)
                                                                    • PCA to find clusters (7)
                                                                    • Recall Yeast Cell Cycle Data
                                                                    • Yeast Cell Cycle Data FDA View
                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                    • Frequency 2 Analysis
                                                                    • Frequency 2 Analysis (2)
                                                                    • Frequency 2 Analysis (3)
                                                                    • Yeast Cell Cycle
                                                                    • SiZer Study of Distrsquon of Angles
                                                                    • Reclassification of Major Genes
                                                                    • Compare to Previous Classifrsquon
                                                                    • New Subpopulation View
                                                                    • New Subpopulation View (2)
                                                                    • Detailed Look at PCA
                                                                    • PCA Rediscovery ndash Renaming
                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                    • An Interesting Historical Note
                                                                    • An Interesting Historical Note (2)
                                                                    • Detailed Look at PCA (2)
                                                                    • Detailed Look at PCA (3)
                                                                    • Review of Linear Algebra
                                                                    • Review of Linear Algebra (2)
                                                                    • Review of Linear Algebra (3)
                                                                    • Review of Linear Algebra (Cont)
                                                                    • Review of Linear Algebra (Cont) (2)
                                                                    • Review of Linear Algebra (Cont) (3)
                                                                    • Review of Linear Algebra (Cont) (4)
                                                                    • Review of Linear Algebra (Cont) (5)
                                                                    • Review of Linear Algebra (Cont) (6)
                                                                    • Review of Linear Algebra (Cont) (7)
                                                                    • Review of Linear Algebra (Cont) (8)
                                                                    • Review of Linear Algebra (Cont) (9)
                                                                    • Review of Linear Algebra (Cont) (10)
                                                                    • Review of Linear Algebra (Cont) (11)
                                                                    • Review of Linear Algebra (Cont) (12)
                                                                    • Review of Linear Algebra (Cont) (13)
                                                                    • Review of Linear Algebra (Cont) (14)
                                                                    • Review of Linear Algebra (Cont) (15)
                                                                    • Review of Linear Algebra (Cont) (16)
                                                                    • Review of Linear Algebra (Cont) (17)
                                                                    • Review of Linear Algebra (Cont) (18)
                                                                    • Review of Linear Algebra (Cont) (19)
                                                                    • Review of Linear Algebra (Cont) (20)
                                                                    • Review of Linear Algebra (Cont) (21)
                                                                    • Review of Linear Algebra (Cont) (22)
                                                                    • Review of Linear Algebra (Cont) (23)
                                                                    • Review of Linear Algebra (Cont) (24)
                                                                    • Review of Linear Algebra (Cont) (25)
                                                                    • Review of Linear Algebra (Cont) (26)
                                                                    • Review of Linear Algebra (Cont) (27)
                                                                    • Review of Linear Algebra (Cont) (28)
                                                                    • Review of Linear Algebra (Cont) (29)
                                                                    • Review of Linear Algebra (Cont) (30)
                                                                    • Review of Linear Algebra (Cont) (31)
                                                                    • Review of Linear Algebra (Cont) (32)
                                                                    • Review of Linear Algebra (Cont) (33)
                                                                    • Review of Linear Algebra (Cont) (34)
                                                                    • Review of Linear Algebra (Cont) (35)
                                                                    • Review of Linear Algebra (Cont) (36)
                                                                    • Review of Linear Algebra (Cont) (37)
                                                                    • Review of Linear Algebra (Cont) (38)
                                                                    • Review of Linear Algebra (Cont) (39)
                                                                    • Review of Linear Algebra (Cont) (40)
                                                                    • Review of Linear Algebra (Cont) (41)
                                                                    • Review of Linear Algebra (Cont) (42)
                                                                    • Review of Linear Algebra (Cont) (43)
                                                                    • Review of Linear Algebra (Cont) (44)
                                                                    • Review of Linear Algebra (Cont) (45)
                                                                    • Review of Linear Algebra (Cont) (46)
                                                                    • Review of Linear Algebra (Cont) (47)
                                                                    • Review of Linear Algebra (Cont) (48)
                                                                    • Review of Linear Algebra (Cont) (49)
                                                                    • Review of Linear Algebra (Cont) (50)
                                                                    • Review of Linear Algebra (Cont) (51)
                                                                    • Review of Linear Algebra (Cont) (52)
                                                                    • Review of Linear Algebra (Cont) (53)
                                                                    • Review of Linear Algebra (Cont) (54)
                                                                    • Review of Linear Algebra (Cont) (55)
                                                                    • Review of Linear Algebra (Cont) (56)
                                                                    • Review of Linear Algebra (Cont) (57)
                                                                    • Review of Linear Algebra (Cont) (58)
                                                                    • Review of Linear Algebra (Cont) (59)
                                                                    • Review of Linear Algebra (Cont) (60)
                                                                    • Review of Linear Algebra (Cont) (61)
                                                                    • Review of Linear Algebra (Cont) (62)
                                                                    • Review of Linear Algebra (Cont) (63)
                                                                    • Review of Linear Algebra (Cont) (64)
                                                                    • Review of Linear Algebra (Cont) (65)
                                                                    • Review of Linear Algebra (Cont) (66)
                                                                    • Review of Linear Algebra (Cont) (67)
                                                                    • Review of Linear Algebra (Cont) (68)
                                                                    • Review of Linear Algebra (Cont) (69)
                                                                    • Review of Linear Algebra (Cont) (70)
                                                                    • Review of Linear Algebra (Cont) (71)
                                                                    • Review of Linear Algebra (Cont) (72)
                                                                    • Review of Linear Algebra (Cont) (73)
                                                                    • Review of Linear Algebra (Cont) (74)
                                                                    • Review of Linear Algebra (Cont) (75)
                                                                    • Review of Linear Algebra (Cont) (76)
                                                                    • Review of Linear Algebra (Cont) (77)
                                                                    • Review of Linear Algebra (Cont) (78)
                                                                    • Review of Linear Algebra (Cont) (79)
                                                                    • Review of Linear Algebra (Cont) (80)
                                                                    • Review of Linear Algebra (Cont) (81)

                                                                      PCA Rediscovery ndash Renaming

                                                                      Statistics Principal Component Analysis (PCA)

                                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                                      PCA Rediscovery ndash Renaming

                                                                      Statistics Principal Component Analysis (PCA)

                                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                                      PCA Rediscovery ndash Renaming

                                                                      Statistics Principal Component Analysis (PCA)

                                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                      PCA Rediscovery ndash Renaming

                                                                      Statistics Principal Component Analysis (PCA)

                                                                      Social Sciences Factor Analysis (PCA is a subset)

                                                                      Probability Electrical EngKarhunen ndash Loeve expansion

                                                                      Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                      Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                      An Interesting Historical Note

                                                                      The 1st () application of PCA to Functional

                                                                      Data Analysis

                                                                      An Interesting Historical Note

                                                                      The 1st () application of PCA to Functional

                                                                      Data Analysis

                                                                      Rao (1958)

                                                                      1st Paper with ldquoCurves as Data Objectsrdquo

                                                                      viewpoint

                                                                      Detailed Look at PCA

                                                                      Three Important (amp Interesting) Viewpoints

                                                                      1 Mathematics

                                                                      2 Numerics

                                                                      3 Statistics

                                                                      Goal Study Interrelationships

                                                                      Detailed Look at PCA

                                                                      Three Important (amp Interesting) Viewpoints

                                                                      1 Mathematics

                                                                      2 Numerics

                                                                      3 Statistics

                                                                      1st Review Linear Alg and Multivar Prob

                                                                      Review of Linear Algebra

                                                                      Vector Space

                                                                      bull set of ldquovectorsrdquo

                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                      xa

                                                                      Review of Linear Algebra

                                                                      Vector Space

                                                                      bull set of ldquovectorsrdquo

                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                      ( in space)

                                                                      xa

                                                                      i

                                                                      ii xa

                                                                      Review of Linear Algebra

                                                                      Vector Space

                                                                      bull set of ldquovectorsrdquo

                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                      ( in space)

                                                                      eg

                                                                      ldquo dim Euclidrsquon spacerdquo

                                                                      xa

                                                                      i

                                                                      ii xa

                                                                      d

                                                                      d

                                                                      d xx

                                                                      x

                                                                      x

                                                                      x 1

                                                                      1

                                                                      d

                                                                      Review of Linear Algebra (Cont)

                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                      Review of Linear Algebra (Cont)

                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                      Review of Linear Algebra (Cont)

                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                      Note Planes not Through the Origin

                                                                      are not Subspaces

                                                                      (Do not Contain )00 x

                                                                      Review of Linear Algebra (Cont)

                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                      (all Linear Combos of them =

                                                                      = Containing Hyperplane

                                                                      through Origin)

                                                                      Review of Linear Algebra (Cont)

                                                                      Basis of Subspace Set of Vectors that

                                                                      bull Span ie Everything is a Lin Com of them

                                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                      Review of Linear Algebra (Cont)

                                                                      Basis of Subspace Set of Vectors that

                                                                      bull Span ie Everything is a Lin Com of them

                                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                      bull eg ldquoUnit Vector Basisrdquo

                                                                      bull Since

                                                                      d

                                                                      1

                                                                      0

                                                                      0

                                                                      0

                                                                      1

                                                                      0

                                                                      0

                                                                      0

                                                                      1

                                                                      1

                                                                      0

                                                                      0

                                                                      0

                                                                      1

                                                                      0

                                                                      0

                                                                      0

                                                                      1

                                                                      212

                                                                      1

                                                                      d

                                                                      d

                                                                      xxx

                                                                      x

                                                                      x

                                                                      x

                                                                      Review of Linear Algebra (Cont)

                                                                      Basis Matrix of subspace of

                                                                      Given a basis

                                                                      create matrix of columns

                                                                      dnvv 1

                                                                      nddnd

                                                                      n

                                                                      n

                                                                      vv

                                                                      vv

                                                                      vvB

                                                                      1

                                                                      111

                                                                      1

                                                                      Review of Linear Algebra (Cont)

                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                      where

                                                                      n

                                                                      iii aBva

                                                                      1

                                                                      na

                                                                      a

                                                                      a 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                      where

                                                                      Note Right Multiplication Gives

                                                                      Linear Combination of Column Vectors

                                                                      n

                                                                      iii aBva

                                                                      1

                                                                      na

                                                                      a

                                                                      a 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                      where

                                                                      Check sizes

                                                                      n

                                                                      iii aBva

                                                                      1

                                                                      na

                                                                      a

                                                                      a 1

                                                                      )1()(1 nndd

                                                                      Review of Linear Algebra (Cont)

                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                      For matrices

                                                                      Define the Matrix Product

                                                                      mkk

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      nmm

                                                                      n

                                                                      bb

                                                                      bb

                                                                      B

                                                                      1

                                                                      111

                                                                      m

                                                                      iniik

                                                                      m

                                                                      iiik

                                                                      m

                                                                      inii

                                                                      m

                                                                      iii

                                                                      baba

                                                                      baba

                                                                      AB

                                                                      1

                                                                      11

                                                                      11

                                                                      111

                                                                      Review of Linear Algebra (Cont)

                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                      For matrices

                                                                      Define the Matrix Product

                                                                      (Inner Products of Rows With Columns )

                                                                      mkk

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      nmm

                                                                      n

                                                                      bb

                                                                      bb

                                                                      B

                                                                      1

                                                                      111

                                                                      m

                                                                      iniik

                                                                      m

                                                                      iiik

                                                                      m

                                                                      inii

                                                                      m

                                                                      iii

                                                                      baba

                                                                      baba

                                                                      AB

                                                                      1

                                                                      11

                                                                      11

                                                                      111

                                                                      A B

                                                                      Review of Linear Algebra (Cont)

                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                      For matrices

                                                                      Define the Matrix Product

                                                                      (Inner Products of Rows With Columns )

                                                                      (Composition of Linear Transformations)

                                                                      mkk

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      nmm

                                                                      n

                                                                      bb

                                                                      bb

                                                                      B

                                                                      1

                                                                      111

                                                                      m

                                                                      iniik

                                                                      m

                                                                      iiik

                                                                      m

                                                                      inii

                                                                      m

                                                                      iii

                                                                      baba

                                                                      baba

                                                                      AB

                                                                      1

                                                                      11

                                                                      11

                                                                      111

                                                                      A B

                                                                      Review of Linear Algebra (Cont)

                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                      For matrices

                                                                      Define the Matrix Product

                                                                      (Inner Products of Rows With Columns )

                                                                      (Composition of Linear Transformations)

                                                                      Often Useful to Check Sizes

                                                                      mkk

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      nmm

                                                                      n

                                                                      bb

                                                                      bb

                                                                      B

                                                                      1

                                                                      111

                                                                      m

                                                                      iniik

                                                                      m

                                                                      iiik

                                                                      m

                                                                      inii

                                                                      m

                                                                      iii

                                                                      baba

                                                                      baba

                                                                      AB

                                                                      1

                                                                      11

                                                                      11

                                                                      111

                                                                      nmmknk

                                                                      A B

                                                                      Review of Linear Algebra (Cont)

                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                      For matrices

                                                                      Define the Matrix Product

                                                                      (Inner Products of Rows With Columns )

                                                                      (Composition of Linear Transformations)

                                                                      Often Useful to Check Sizes

                                                                      mkk

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      nmm

                                                                      n

                                                                      bb

                                                                      bb

                                                                      B

                                                                      1

                                                                      111

                                                                      m

                                                                      iniik

                                                                      m

                                                                      iiik

                                                                      m

                                                                      inii

                                                                      m

                                                                      iii

                                                                      baba

                                                                      baba

                                                                      AB

                                                                      1

                                                                      11

                                                                      11

                                                                      111

                                                                      nmmknk

                                                                      A B

                                                                      Review of Linear Algebra (Cont)

                                                                      Matrix Trace

                                                                      bull For a Square Matrix

                                                                      bull Define

                                                                      mmm

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      m

                                                                      iiiaAtr

                                                                      1)(

                                                                      Review of Linear Algebra (Cont)

                                                                      Matrix Trace

                                                                      bull For a Square Matrix

                                                                      bull Define

                                                                      bull Trace Commutes with Matrix Multiplication

                                                                      mmm

                                                                      m

                                                                      aa

                                                                      aa

                                                                      A

                                                                      1

                                                                      111

                                                                      m

                                                                      iiiaAtr

                                                                      1)(

                                                                      BAtrABtr

                                                                      Review of Linear Algebra (Cont)

                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                      bull Number of Elements in a Basis (Unique)

                                                                      Review of Linear Algebra (Cont)

                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                      bull Number of Elements in a Basis (Unique)

                                                                      bull (Use Basis Above)

                                                                      bull eg dim of a line is 1

                                                                      bull eg dim of a plane is 2

                                                                      dd dim

                                                                      Review of Linear Algebra (Cont)

                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                      bull Number of Elements in a Basis (Unique)

                                                                      bull (Use Basis Above)

                                                                      bull eg dim of a line is 1

                                                                      bull eg dim of a plane is 2

                                                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                                                      (in Statistical Uses eg ANOVA)

                                                                      dd dim

                                                                      Review of Linear Algebra (Cont)

                                                                      Norm of a Vector

                                                                      bull in d 21

                                                                      21

                                                                      1

                                                                      2 xxxx td

                                                                      jj

                                                                      Review of Linear Algebra (Cont)

                                                                      Norm of a Vector

                                                                      bull in

                                                                      bull Idea length of the vector

                                                                      d 2121

                                                                      1

                                                                      2 xxxx td

                                                                      jj

                                                                      Review of Linear Algebra (Cont)

                                                                      Norm of a Vector

                                                                      bull in

                                                                      bull Idea length of the vector

                                                                      bull Note strange properties for high

                                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                                      d 2121

                                                                      1

                                                                      2 xxxx td

                                                                      jj

                                                                      d

                                                                      d

                                                                      Review of Linear Algebra (Cont)

                                                                      Norm of a Vector (cont)

                                                                      bull Length Normalized Vector

                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                      amp is a Direction Vector)

                                                                      x

                                                                      x

                                                                      Review of Linear Algebra (Cont)

                                                                      Norm of a Vector (cont)

                                                                      bull Length Normalized Vector

                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                      amp is a Direction Vector)

                                                                      bull Define Distance as

                                                                      x

                                                                      x

                                                                      yxyxyxyxd t

                                                                      Review of Linear Algebra (Cont)

                                                                      Inner (Dot Scalar) Product

                                                                      bull for Vectors and

                                                                      yxyxyx td

                                                                      jjj

                                                                      1

                                                                      x y

                                                                      Review of Linear Algebra (Cont)

                                                                      Inner (Dot Scalar) Product

                                                                      bull for Vectors and

                                                                      bull Related to Norm via

                                                                      yxyxyx td

                                                                      jjj

                                                                      1

                                                                      x y

                                                                      21 xxx

                                                                      Review of Linear Algebra (Cont)

                                                                      Inner (Dot Scalar) Product (cont)

                                                                      bull measures ldquoangle between and rdquo as

                                                                      yyxx

                                                                      yx

                                                                      yx

                                                                      yxyxangle

                                                                      tt

                                                                      t

                                                                      11 cos

                                                                      cos

                                                                      x y

                                                                      Review of Linear Algebra (Cont)

                                                                      Inner (Dot Scalar) Product (cont)

                                                                      bull measures ldquoangle between and rdquo as

                                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                                      if and only if

                                                                      yyxx

                                                                      yx

                                                                      yx

                                                                      yxyxangle

                                                                      tt

                                                                      t

                                                                      11 cos

                                                                      cos

                                                                      x y

                                                                      yx 0 yx

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis

                                                                      bull All Orthogonal to each other

                                                                      ie for

                                                                      bull All have Length 1

                                                                      ie for

                                                                      nvv 1

                                                                      1 ii vv

                                                                      0 ii vv ii

                                                                      ni 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis (cont)

                                                                      bull Spectral Representation

                                                                      where

                                                                      nvv 1

                                                                      n

                                                                      iii vax

                                                                      1

                                                                      ii vxa

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis (cont)

                                                                      bull Spectral Representation

                                                                      where

                                                                      (Coefficient is Inner Product Cool Notation)

                                                                      nvv 1

                                                                      n

                                                                      iii vax

                                                                      1

                                                                      ii vxa

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis (cont)

                                                                      bull Spectral Representation

                                                                      where

                                                                      Check

                                                                      nvv 1

                                                                      n

                                                                      iii vax

                                                                      1

                                                                      ii vxa

                                                                      iii

                                                                      n

                                                                      iii

                                                                      n

                                                                      iiii avvavvavx

                                                                      1

                                                                      1

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis (cont)

                                                                      bull Spectral Representation

                                                                      where

                                                                      Check

                                                                      bull Matrix Notation where ie

                                                                      For the Basis Matrix

                                                                      nvv 1

                                                                      n

                                                                      iii vax

                                                                      1

                                                                      ii vxa

                                                                      iii

                                                                      n

                                                                      iii

                                                                      n

                                                                      iiii avvavvavx

                                                                      1

                                                                      1

                                                                      aBx Bxa tt xBa t

                                                                      nvvB 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Orthonormal Basis (cont)

                                                                      bull Spectral Representation

                                                                      where

                                                                      Check

                                                                      bull Matrix Notation where ie

                                                                      is called transform of

                                                                      (eg Fourier or Wavelet)

                                                                      nvv 1

                                                                      n

                                                                      iii vax

                                                                      1

                                                                      ii vxa

                                                                      iii

                                                                      n

                                                                      iii

                                                                      n

                                                                      iiii avvavvavx

                                                                      1

                                                                      1

                                                                      aBx Bxa tt xBa t

                                                                      xa

                                                                      Review of Linear Algebra (Cont)

                                                                      Parseval identity for

                                                                      in subsp genrsquod by o n basis

                                                                      x

                                                                      nvv 1

                                                                      2

                                                                      1

                                                                      22

                                                                      1

                                                                      2 aavxx

                                                                      n

                                                                      ii

                                                                      n

                                                                      ii

                                                                      Review of Linear Algebra (Cont)

                                                                      Parseval identity for

                                                                      in subsp genrsquod by o n basis

                                                                      bull Pythagorean theorem

                                                                      bull ldquoDecomposition of Energyrdquo

                                                                      bull ANOVA - sums of squares

                                                                      x

                                                                      nvv 1

                                                                      2

                                                                      1

                                                                      22

                                                                      1

                                                                      2 aavxx

                                                                      n

                                                                      ii

                                                                      n

                                                                      ii

                                                                      Review of Linear Algebra (Cont)

                                                                      Parseval identity for

                                                                      in subsp genrsquod by o n basis

                                                                      bull Pythagorean theorem

                                                                      bull ldquoDecomposition of Energyrdquo

                                                                      bull ANOVA - sums of squares

                                                                      bull Transform has same length as

                                                                      ie ldquorotation in rdquo

                                                                      x

                                                                      nvv 1

                                                                      2

                                                                      1

                                                                      22

                                                                      1

                                                                      2 aavxx

                                                                      n

                                                                      ii

                                                                      n

                                                                      ii

                                                                      a xd

                                                                      Projection of a Vector onto a Subspace

                                                                      bull Idea Member of that is Closest to

                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                      Review of Linear Algebra (Cont)x

                                                                      xV

                                                                      V

                                                                      Projection of a Vector onto a Subspace

                                                                      bull Idea Member of that is Closest to

                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                      bull Find that Solves

                                                                      (ldquoLeast Squaresrdquo)

                                                                      Review of Linear Algebra (Cont)x

                                                                      xV

                                                                      V

                                                                      VxPV vxVv

                                                                      min

                                                                      Projection of a Vector onto a Subspace

                                                                      bull Idea Member of that is Closest to

                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                      bull Find that Solves

                                                                      (ldquoLeast Squaresrdquo)

                                                                      bull For Inner Product (Hilbert) Space

                                                                      Exists and is Unique

                                                                      Review of Linear Algebra (Cont)x

                                                                      xV

                                                                      V

                                                                      VxPV vxVv

                                                                      min

                                                                      xPV

                                                                      Projection of a Vector onto a Subspace (cont)

                                                                      bull General Solution in for Basis Matrix

                                                                      Review of Linear Algebra (Cont)

                                                                      d VB

                                                                      xBBBBxP tVV

                                                                      tVVV

                                                                      1

                                                                      Projection of a Vector onto a Subspace (cont)

                                                                      bull General Solution in for Basis Matrix

                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                      (thus projection is another linear operation)

                                                                      Review of Linear Algebra (Cont)

                                                                      d VB

                                                                      xBBBBxP tVV

                                                                      tVVV

                                                                      1

                                                                      tVV

                                                                      tVVV BBBBP

                                                                      1

                                                                      Projection of a Vector onto a Subspace (cont)

                                                                      bull General Solution in for Basis Matrix

                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                      (thus projection is another linear operation)

                                                                      (note same operation underlies least squares)

                                                                      Review of Linear Algebra (Cont)

                                                                      d VB

                                                                      xBBBBxP tVV

                                                                      tVVV

                                                                      1

                                                                      tVV

                                                                      tVVV BBBBP

                                                                      1

                                                                      Review of Linear Algebra (Cont)

                                                                      Projection using Orthonormal Basis

                                                                      bull Basis Matrix is Orthonormal

                                                                      nnVtV IBB

                                                                      10

                                                                      01

                                                                      1

                                                                      111

                                                                      1

                                                                      1

                                                                      nnn

                                                                      n

                                                                      ntn

                                                                      t

                                                                      vvvv

                                                                      vvvv

                                                                      vv

                                                                      v

                                                                      v

                                                                      nvv 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Projection using Orthonormal Basis

                                                                      bull Basis Matrix is Orthonormal

                                                                      bull So =

                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                      (Recall Right Multrsquon)

                                                                      nnVtV IBB

                                                                      10

                                                                      01

                                                                      1

                                                                      111

                                                                      1

                                                                      1

                                                                      nnn

                                                                      n

                                                                      ntn

                                                                      t

                                                                      vvvv

                                                                      vvvv

                                                                      vv

                                                                      v

                                                                      v

                                                                      xBBxP tVVV

                                                                      x V

                                                                      nvv 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Projection using Orthonormal Basis (cont)

                                                                      bull For Orthogonal Complement

                                                                      and

                                                                      V

                                                                      xPxPx VV 222xPxPx VV

                                                                      Review of Linear Algebra (Cont)

                                                                      Projection using Orthonormal Basis (cont)

                                                                      bull For Orthogonal Complement

                                                                      and

                                                                      bull Parseval Inequality

                                                                      V

                                                                      xPxPx VV 222xPxPx VV

                                                                      2

                                                                      1

                                                                      22

                                                                      1

                                                                      22 aavxxxP

                                                                      n

                                                                      ii

                                                                      n

                                                                      iiV

                                                                      Review of Linear Algebra (Cont)

                                                                      (Real) Unitary Matrices with

                                                                      bull Orthonormal Basis Matrix

                                                                      (So All of Above Applies)

                                                                      ddU IUU t

                                                                      Review of Linear Algebra (Cont)

                                                                      (Real) Unitary Matrices with

                                                                      bull Orthonormal Basis Matrix

                                                                      (So All of Above Applies)

                                                                      bull Note Transformrsquon is Distance Preserving

                                                                      ddU IUU t

                                                                      yxdyxyxyxUyUxUdn

                                                                      i ii 2

                                                                      1

                                                                      Review of Linear Algebra (Cont)

                                                                      (Real) Unitary Matrices with

                                                                      bull Orthonormal Basis Matrix

                                                                      (So All of Above Applies)

                                                                      bull Note Transformrsquon is Distance Preserving

                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                      ddU IUU t

                                                                      yxdyxyxyxUyUxUdn

                                                                      i ii 2

                                                                      1

                                                                      U

                                                                      Review of Linear Algebra (Cont)

                                                                      Singular Value Decomposition (SVD)

                                                                      For a Matrix

                                                                      Find

                                                                      ndX

                                                                      Review of Linear Algebra (Cont)

                                                                      Singular Value Decomposition (SVD)

                                                                      For a Matrix

                                                                      Find a Diagonal Matrix

                                                                      with Entries

                                                                      called Singular Values

                                                                      ndX

                                                                      ndS

                                                                      )min(1 ndss

                                                                      Review of Linear Algebra (Cont)

                                                                      Singular Value Decomposition (SVD)

                                                                      For a Matrix

                                                                      Find a Diagonal Matrix

                                                                      with Entries

                                                                      called Singular Values

                                                                      And Unitary (Rotation) Matrices

                                                                      (recall )

                                                                      ndX

                                                                      ndS

                                                                      )min(1 ndss

                                                                      ddU nnV

                                                                      IVVUU tt

                                                                      Review of Linear Algebra (Cont)

                                                                      Singular Value Decomposition (SVD)

                                                                      For a Matrix

                                                                      Find a Diagonal Matrix

                                                                      with Entries

                                                                      called Singular Values

                                                                      And Unitary (Rotation) Matrices

                                                                      (recall )

                                                                      So That

                                                                      ndX

                                                                      ndS

                                                                      )min(1 ndss

                                                                      ddU nnV

                                                                      IVVUU tt tUSVX

                                                                      Review of Linear Algebra (Cont)

                                                                      Intuition behind Singular Value Decomposition

                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                      vVSUvVSUvX tt

                                                                      Review of Linear Algebra (Cont)

                                                                      Intuition behind Singular Value Decomposition

                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                      bull First rotate

                                                                      X

                                                                      vVSUvVSUvX tt

                                                                      Review of Linear Algebra (Cont)

                                                                      Intuition behind Singular Value Decomposition

                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                      bull First rotate

                                                                      bull Second rescale coordinate axes (by )

                                                                      X

                                                                      vVSUvVSUvX tt

                                                                      is

                                                                      Review of Linear Algebra (Cont)

                                                                      Intuition behind Singular Value Decomposition

                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                      bull First rotate

                                                                      bull Second rescale coordinate axes (by )

                                                                      bull Third rotate again

                                                                      X

                                                                      vVSUvVSUvX tt

                                                                      is

                                                                      Review of Linear Algebra (Cont)

                                                                      Intuition behind Singular Value Decomposition

                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                      bull First rotate

                                                                      bull Second rescale coordinate axes (by )

                                                                      bull Third rotate again

                                                                      bull ie have diagonalized the transformation

                                                                      X

                                                                      vVSUvVSUvX tt

                                                                      is

                                                                      SVD Compact Representation

                                                                      Useful Labeling

                                                                      Singular Values in Increasing Order

                                                                      Review of Linear Algebra (Cont)

                                                                      )min(1 dnss

                                                                      SVD Compact Representation

                                                                      Useful Labeling

                                                                      Singular Values in Increasing Order

                                                                      Note singular values = 0 can be omitted

                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                      Review of Linear Algebra (Cont)

                                                                      )min(1 dnss

                                                                      Review of Linear Algebra (Cont)

                                                                      r

                                                                      SVD Compact Representation

                                                                      Useful Labeling

                                                                      Singular Values in Increasing Order

                                                                      Note singular values = 0 can be omitted

                                                                      Let = of positive singular values

                                                                      )min(1 dnss

                                                                      Review of Linear Algebra (Cont)

                                                                      r

                                                                      SVD Compact Representation

                                                                      Useful Labeling

                                                                      Singular Values in Increasing Order

                                                                      Note singular values = 0 can be omitted

                                                                      Let = of positive singular values

                                                                      Then

                                                                      Where are truncations of

                                                                      trnrrrd VSUX

                                                                      VSU

                                                                      )min(1 dnss

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Full Representation

                                                                      =

                                                                      Graphics Display Assumes

                                                                      ndX ddU ndS nn

                                                                      tV

                                                                      nd

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Full Representation

                                                                      =

                                                                      Full Rank Basis Matrix

                                                                      ndX ddU ndS nn

                                                                      tV

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Full Representation

                                                                      =

                                                                      Full Rank Basis Matrix

                                                                      All 0s in Bottom

                                                                      ndX ddU ndS nn

                                                                      tV

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Reduced Representation

                                                                      =

                                                                      These Columns Get 0ed Out

                                                                      ndX ddU nnS nn

                                                                      tV

                                                                      nnd 0

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Reduced Representation

                                                                      =ndX ndU nnS nn

                                                                      tV

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Reduced Representation

                                                                      =

                                                                      Also Some of These May be 0

                                                                      ndX ndU nnS nn

                                                                      tV

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Compact Representation

                                                                      = ndX rdU

                                                                      rrS nrtV

                                                                      0

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Compact Representation

                                                                      =

                                                                      These Get 0ed Out

                                                                      ndX rdU

                                                                      rrS nrtV

                                                                      0

                                                                      Review of Linear Algebra (Cont)

                                                                      SVD Compact Representation

                                                                      = ndX rdU

                                                                      rrS nrtV

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition

                                                                      For a (Symmetric) Square Matrix

                                                                      Find

                                                                      ddX

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition

                                                                      For a (Symmetric) Square Matrix

                                                                      Find a Diagonal Matrix

                                                                      Called Eigenvalues

                                                                      Convenient Ordering

                                                                      ddX

                                                                      d

                                                                      D

                                                                      0

                                                                      01

                                                                      n 1

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition

                                                                      For a (Symmetric) Square Matrix

                                                                      Find a Diagonal Matrix

                                                                      And an Orthonormal Matrix

                                                                      (ie )

                                                                      ddX

                                                                      d

                                                                      D

                                                                      0

                                                                      01

                                                                      ddB

                                                                      ddtt IBBBB

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition

                                                                      For a (Symmetric) Square Matrix

                                                                      Find a Diagonal Matrix

                                                                      And an Orthonormal Matrix

                                                                      (ie )

                                                                      So that ie

                                                                      ddX

                                                                      d

                                                                      D

                                                                      0

                                                                      01

                                                                      ddB

                                                                      ddtt IBBBB

                                                                      DBBX tBDBX

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                      (looks similar)

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                      Complex (uses )

                                                                      VU

                                                                      1i

                                                                      Review of Linear Algebra (Cont)

                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                      VU

                                                                      X

                                                                      BVU

                                                                      1i

                                                                      • Participant Presentations
                                                                      • PCA to find clusters
                                                                      • Statistical Smoothing
                                                                      • Kernel Density Estimation
                                                                      • Scatterplot Smoothing
                                                                      • Statistical Smoothing (2)
                                                                      • SiZer Background
                                                                      • SiZer Background (2)
                                                                      • SiZer Background (3)
                                                                      • SiZer Background (4)
                                                                      • SiZer Background (5)
                                                                      • SiZer Overview
                                                                      • PCA to find clusters (2)
                                                                      • PCA to find clusters (3)
                                                                      • PCA to find clusters (4)
                                                                      • PCA to find clusters (5)
                                                                      • PCA to find clusters (6)
                                                                      • PCA to find clusters (7)
                                                                      • Recall Yeast Cell Cycle Data
                                                                      • Yeast Cell Cycle Data FDA View
                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                      • Frequency 2 Analysis
                                                                      • Frequency 2 Analysis (2)
                                                                      • Frequency 2 Analysis (3)
                                                                      • Yeast Cell Cycle
                                                                      • SiZer Study of Distrsquon of Angles
                                                                      • Reclassification of Major Genes
                                                                      • Compare to Previous Classifrsquon
                                                                      • New Subpopulation View
                                                                      • New Subpopulation View (2)
                                                                      • Detailed Look at PCA
                                                                      • PCA Rediscovery ndash Renaming
                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                      • An Interesting Historical Note
                                                                      • An Interesting Historical Note (2)
                                                                      • Detailed Look at PCA (2)
                                                                      • Detailed Look at PCA (3)
                                                                      • Review of Linear Algebra
                                                                      • Review of Linear Algebra (2)
                                                                      • Review of Linear Algebra (3)
                                                                      • Review of Linear Algebra (Cont)
                                                                      • Review of Linear Algebra (Cont) (2)
                                                                      • Review of Linear Algebra (Cont) (3)
                                                                      • Review of Linear Algebra (Cont) (4)
                                                                      • Review of Linear Algebra (Cont) (5)
                                                                      • Review of Linear Algebra (Cont) (6)
                                                                      • Review of Linear Algebra (Cont) (7)
                                                                      • Review of Linear Algebra (Cont) (8)
                                                                      • Review of Linear Algebra (Cont) (9)
                                                                      • Review of Linear Algebra (Cont) (10)
                                                                      • Review of Linear Algebra (Cont) (11)
                                                                      • Review of Linear Algebra (Cont) (12)
                                                                      • Review of Linear Algebra (Cont) (13)
                                                                      • Review of Linear Algebra (Cont) (14)
                                                                      • Review of Linear Algebra (Cont) (15)
                                                                      • Review of Linear Algebra (Cont) (16)
                                                                      • Review of Linear Algebra (Cont) (17)
                                                                      • Review of Linear Algebra (Cont) (18)
                                                                      • Review of Linear Algebra (Cont) (19)
                                                                      • Review of Linear Algebra (Cont) (20)
                                                                      • Review of Linear Algebra (Cont) (21)
                                                                      • Review of Linear Algebra (Cont) (22)
                                                                      • Review of Linear Algebra (Cont) (23)
                                                                      • Review of Linear Algebra (Cont) (24)
                                                                      • Review of Linear Algebra (Cont) (25)
                                                                      • Review of Linear Algebra (Cont) (26)
                                                                      • Review of Linear Algebra (Cont) (27)
                                                                      • Review of Linear Algebra (Cont) (28)
                                                                      • Review of Linear Algebra (Cont) (29)
                                                                      • Review of Linear Algebra (Cont) (30)
                                                                      • Review of Linear Algebra (Cont) (31)
                                                                      • Review of Linear Algebra (Cont) (32)
                                                                      • Review of Linear Algebra (Cont) (33)
                                                                      • Review of Linear Algebra (Cont) (34)
                                                                      • Review of Linear Algebra (Cont) (35)
                                                                      • Review of Linear Algebra (Cont) (36)
                                                                      • Review of Linear Algebra (Cont) (37)
                                                                      • Review of Linear Algebra (Cont) (38)
                                                                      • Review of Linear Algebra (Cont) (39)
                                                                      • Review of Linear Algebra (Cont) (40)
                                                                      • Review of Linear Algebra (Cont) (41)
                                                                      • Review of Linear Algebra (Cont) (42)
                                                                      • Review of Linear Algebra (Cont) (43)
                                                                      • Review of Linear Algebra (Cont) (44)
                                                                      • Review of Linear Algebra (Cont) (45)
                                                                      • Review of Linear Algebra (Cont) (46)
                                                                      • Review of Linear Algebra (Cont) (47)
                                                                      • Review of Linear Algebra (Cont) (48)
                                                                      • Review of Linear Algebra (Cont) (49)
                                                                      • Review of Linear Algebra (Cont) (50)
                                                                      • Review of Linear Algebra (Cont) (51)
                                                                      • Review of Linear Algebra (Cont) (52)
                                                                      • Review of Linear Algebra (Cont) (53)
                                                                      • Review of Linear Algebra (Cont) (54)
                                                                      • Review of Linear Algebra (Cont) (55)
                                                                      • Review of Linear Algebra (Cont) (56)
                                                                      • Review of Linear Algebra (Cont) (57)
                                                                      • Review of Linear Algebra (Cont) (58)
                                                                      • Review of Linear Algebra (Cont) (59)
                                                                      • Review of Linear Algebra (Cont) (60)
                                                                      • Review of Linear Algebra (Cont) (61)
                                                                      • Review of Linear Algebra (Cont) (62)
                                                                      • Review of Linear Algebra (Cont) (63)
                                                                      • Review of Linear Algebra (Cont) (64)
                                                                      • Review of Linear Algebra (Cont) (65)
                                                                      • Review of Linear Algebra (Cont) (66)
                                                                      • Review of Linear Algebra (Cont) (67)
                                                                      • Review of Linear Algebra (Cont) (68)
                                                                      • Review of Linear Algebra (Cont) (69)
                                                                      • Review of Linear Algebra (Cont) (70)
                                                                      • Review of Linear Algebra (Cont) (71)
                                                                      • Review of Linear Algebra (Cont) (72)
                                                                      • Review of Linear Algebra (Cont) (73)
                                                                      • Review of Linear Algebra (Cont) (74)
                                                                      • Review of Linear Algebra (Cont) (75)
                                                                      • Review of Linear Algebra (Cont) (76)
                                                                      • Review of Linear Algebra (Cont) (77)
                                                                      • Review of Linear Algebra (Cont) (78)
                                                                      • Review of Linear Algebra (Cont) (79)
                                                                      • Review of Linear Algebra (Cont) (80)
                                                                      • Review of Linear Algebra (Cont) (81)

                                                                        PCA Rediscovery ndash Renaming

                                                                        Statistics Principal Component Analysis (PCA)

                                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                                        PCA Rediscovery ndash Renaming

                                                                        Statistics Principal Component Analysis (PCA)

                                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                        PCA Rediscovery ndash Renaming

                                                                        Statistics Principal Component Analysis (PCA)

                                                                        Social Sciences Factor Analysis (PCA is a subset)

                                                                        Probability Electrical EngKarhunen ndash Loeve expansion

                                                                        Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                        Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                        An Interesting Historical Note

                                                                        The 1st () application of PCA to Functional

                                                                        Data Analysis

                                                                        An Interesting Historical Note

                                                                        The 1st () application of PCA to Functional

                                                                        Data Analysis

                                                                        Rao (1958)

                                                                        1st Paper with ldquoCurves as Data Objectsrdquo

                                                                        viewpoint

                                                                        Detailed Look at PCA

                                                                        Three Important (amp Interesting) Viewpoints

                                                                        1 Mathematics

                                                                        2 Numerics

                                                                        3 Statistics

                                                                        Goal Study Interrelationships

                                                                        Detailed Look at PCA

                                                                        Three Important (amp Interesting) Viewpoints

                                                                        1 Mathematics

                                                                        2 Numerics

                                                                        3 Statistics

                                                                        1st Review Linear Alg and Multivar Prob

                                                                        Review of Linear Algebra

                                                                        Vector Space

                                                                        bull set of ldquovectorsrdquo

                                                                        bull and ldquoscalarsrdquo (coefficients)

                                                                        xa

                                                                        Review of Linear Algebra

                                                                        Vector Space

                                                                        bull set of ldquovectorsrdquo

                                                                        bull and ldquoscalarsrdquo (coefficients)

                                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                        ( in space)

                                                                        xa

                                                                        i

                                                                        ii xa

                                                                        Review of Linear Algebra

                                                                        Vector Space

                                                                        bull set of ldquovectorsrdquo

                                                                        bull and ldquoscalarsrdquo (coefficients)

                                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                        ( in space)

                                                                        eg

                                                                        ldquo dim Euclidrsquon spacerdquo

                                                                        xa

                                                                        i

                                                                        ii xa

                                                                        d

                                                                        d

                                                                        d xx

                                                                        x

                                                                        x

                                                                        x 1

                                                                        1

                                                                        d

                                                                        Review of Linear Algebra (Cont)

                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                        Review of Linear Algebra (Cont)

                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                        Review of Linear Algebra (Cont)

                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                        Note Planes not Through the Origin

                                                                        are not Subspaces

                                                                        (Do not Contain )00 x

                                                                        Review of Linear Algebra (Cont)

                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                        (all Linear Combos of them =

                                                                        = Containing Hyperplane

                                                                        through Origin)

                                                                        Review of Linear Algebra (Cont)

                                                                        Basis of Subspace Set of Vectors that

                                                                        bull Span ie Everything is a Lin Com of them

                                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                        Review of Linear Algebra (Cont)

                                                                        Basis of Subspace Set of Vectors that

                                                                        bull Span ie Everything is a Lin Com of them

                                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                        bull eg ldquoUnit Vector Basisrdquo

                                                                        bull Since

                                                                        d

                                                                        1

                                                                        0

                                                                        0

                                                                        0

                                                                        1

                                                                        0

                                                                        0

                                                                        0

                                                                        1

                                                                        1

                                                                        0

                                                                        0

                                                                        0

                                                                        1

                                                                        0

                                                                        0

                                                                        0

                                                                        1

                                                                        212

                                                                        1

                                                                        d

                                                                        d

                                                                        xxx

                                                                        x

                                                                        x

                                                                        x

                                                                        Review of Linear Algebra (Cont)

                                                                        Basis Matrix of subspace of

                                                                        Given a basis

                                                                        create matrix of columns

                                                                        dnvv 1

                                                                        nddnd

                                                                        n

                                                                        n

                                                                        vv

                                                                        vv

                                                                        vvB

                                                                        1

                                                                        111

                                                                        1

                                                                        Review of Linear Algebra (Cont)

                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                        where

                                                                        n

                                                                        iii aBva

                                                                        1

                                                                        na

                                                                        a

                                                                        a 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                        where

                                                                        Note Right Multiplication Gives

                                                                        Linear Combination of Column Vectors

                                                                        n

                                                                        iii aBva

                                                                        1

                                                                        na

                                                                        a

                                                                        a 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                        where

                                                                        Check sizes

                                                                        n

                                                                        iii aBva

                                                                        1

                                                                        na

                                                                        a

                                                                        a 1

                                                                        )1()(1 nndd

                                                                        Review of Linear Algebra (Cont)

                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                        For matrices

                                                                        Define the Matrix Product

                                                                        mkk

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        nmm

                                                                        n

                                                                        bb

                                                                        bb

                                                                        B

                                                                        1

                                                                        111

                                                                        m

                                                                        iniik

                                                                        m

                                                                        iiik

                                                                        m

                                                                        inii

                                                                        m

                                                                        iii

                                                                        baba

                                                                        baba

                                                                        AB

                                                                        1

                                                                        11

                                                                        11

                                                                        111

                                                                        Review of Linear Algebra (Cont)

                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                        For matrices

                                                                        Define the Matrix Product

                                                                        (Inner Products of Rows With Columns )

                                                                        mkk

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        nmm

                                                                        n

                                                                        bb

                                                                        bb

                                                                        B

                                                                        1

                                                                        111

                                                                        m

                                                                        iniik

                                                                        m

                                                                        iiik

                                                                        m

                                                                        inii

                                                                        m

                                                                        iii

                                                                        baba

                                                                        baba

                                                                        AB

                                                                        1

                                                                        11

                                                                        11

                                                                        111

                                                                        A B

                                                                        Review of Linear Algebra (Cont)

                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                        For matrices

                                                                        Define the Matrix Product

                                                                        (Inner Products of Rows With Columns )

                                                                        (Composition of Linear Transformations)

                                                                        mkk

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        nmm

                                                                        n

                                                                        bb

                                                                        bb

                                                                        B

                                                                        1

                                                                        111

                                                                        m

                                                                        iniik

                                                                        m

                                                                        iiik

                                                                        m

                                                                        inii

                                                                        m

                                                                        iii

                                                                        baba

                                                                        baba

                                                                        AB

                                                                        1

                                                                        11

                                                                        11

                                                                        111

                                                                        A B

                                                                        Review of Linear Algebra (Cont)

                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                        For matrices

                                                                        Define the Matrix Product

                                                                        (Inner Products of Rows With Columns )

                                                                        (Composition of Linear Transformations)

                                                                        Often Useful to Check Sizes

                                                                        mkk

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        nmm

                                                                        n

                                                                        bb

                                                                        bb

                                                                        B

                                                                        1

                                                                        111

                                                                        m

                                                                        iniik

                                                                        m

                                                                        iiik

                                                                        m

                                                                        inii

                                                                        m

                                                                        iii

                                                                        baba

                                                                        baba

                                                                        AB

                                                                        1

                                                                        11

                                                                        11

                                                                        111

                                                                        nmmknk

                                                                        A B

                                                                        Review of Linear Algebra (Cont)

                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                        For matrices

                                                                        Define the Matrix Product

                                                                        (Inner Products of Rows With Columns )

                                                                        (Composition of Linear Transformations)

                                                                        Often Useful to Check Sizes

                                                                        mkk

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        nmm

                                                                        n

                                                                        bb

                                                                        bb

                                                                        B

                                                                        1

                                                                        111

                                                                        m

                                                                        iniik

                                                                        m

                                                                        iiik

                                                                        m

                                                                        inii

                                                                        m

                                                                        iii

                                                                        baba

                                                                        baba

                                                                        AB

                                                                        1

                                                                        11

                                                                        11

                                                                        111

                                                                        nmmknk

                                                                        A B

                                                                        Review of Linear Algebra (Cont)

                                                                        Matrix Trace

                                                                        bull For a Square Matrix

                                                                        bull Define

                                                                        mmm

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        m

                                                                        iiiaAtr

                                                                        1)(

                                                                        Review of Linear Algebra (Cont)

                                                                        Matrix Trace

                                                                        bull For a Square Matrix

                                                                        bull Define

                                                                        bull Trace Commutes with Matrix Multiplication

                                                                        mmm

                                                                        m

                                                                        aa

                                                                        aa

                                                                        A

                                                                        1

                                                                        111

                                                                        m

                                                                        iiiaAtr

                                                                        1)(

                                                                        BAtrABtr

                                                                        Review of Linear Algebra (Cont)

                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                        bull Number of Elements in a Basis (Unique)

                                                                        Review of Linear Algebra (Cont)

                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                        bull Number of Elements in a Basis (Unique)

                                                                        bull (Use Basis Above)

                                                                        bull eg dim of a line is 1

                                                                        bull eg dim of a plane is 2

                                                                        dd dim

                                                                        Review of Linear Algebra (Cont)

                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                        bull Number of Elements in a Basis (Unique)

                                                                        bull (Use Basis Above)

                                                                        bull eg dim of a line is 1

                                                                        bull eg dim of a plane is 2

                                                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                                                        (in Statistical Uses eg ANOVA)

                                                                        dd dim

                                                                        Review of Linear Algebra (Cont)

                                                                        Norm of a Vector

                                                                        bull in d 21

                                                                        21

                                                                        1

                                                                        2 xxxx td

                                                                        jj

                                                                        Review of Linear Algebra (Cont)

                                                                        Norm of a Vector

                                                                        bull in

                                                                        bull Idea length of the vector

                                                                        d 2121

                                                                        1

                                                                        2 xxxx td

                                                                        jj

                                                                        Review of Linear Algebra (Cont)

                                                                        Norm of a Vector

                                                                        bull in

                                                                        bull Idea length of the vector

                                                                        bull Note strange properties for high

                                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                                        d 2121

                                                                        1

                                                                        2 xxxx td

                                                                        jj

                                                                        d

                                                                        d

                                                                        Review of Linear Algebra (Cont)

                                                                        Norm of a Vector (cont)

                                                                        bull Length Normalized Vector

                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                        amp is a Direction Vector)

                                                                        x

                                                                        x

                                                                        Review of Linear Algebra (Cont)

                                                                        Norm of a Vector (cont)

                                                                        bull Length Normalized Vector

                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                        amp is a Direction Vector)

                                                                        bull Define Distance as

                                                                        x

                                                                        x

                                                                        yxyxyxyxd t

                                                                        Review of Linear Algebra (Cont)

                                                                        Inner (Dot Scalar) Product

                                                                        bull for Vectors and

                                                                        yxyxyx td

                                                                        jjj

                                                                        1

                                                                        x y

                                                                        Review of Linear Algebra (Cont)

                                                                        Inner (Dot Scalar) Product

                                                                        bull for Vectors and

                                                                        bull Related to Norm via

                                                                        yxyxyx td

                                                                        jjj

                                                                        1

                                                                        x y

                                                                        21 xxx

                                                                        Review of Linear Algebra (Cont)

                                                                        Inner (Dot Scalar) Product (cont)

                                                                        bull measures ldquoangle between and rdquo as

                                                                        yyxx

                                                                        yx

                                                                        yx

                                                                        yxyxangle

                                                                        tt

                                                                        t

                                                                        11 cos

                                                                        cos

                                                                        x y

                                                                        Review of Linear Algebra (Cont)

                                                                        Inner (Dot Scalar) Product (cont)

                                                                        bull measures ldquoangle between and rdquo as

                                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                                        if and only if

                                                                        yyxx

                                                                        yx

                                                                        yx

                                                                        yxyxangle

                                                                        tt

                                                                        t

                                                                        11 cos

                                                                        cos

                                                                        x y

                                                                        yx 0 yx

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis

                                                                        bull All Orthogonal to each other

                                                                        ie for

                                                                        bull All have Length 1

                                                                        ie for

                                                                        nvv 1

                                                                        1 ii vv

                                                                        0 ii vv ii

                                                                        ni 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis (cont)

                                                                        bull Spectral Representation

                                                                        where

                                                                        nvv 1

                                                                        n

                                                                        iii vax

                                                                        1

                                                                        ii vxa

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis (cont)

                                                                        bull Spectral Representation

                                                                        where

                                                                        (Coefficient is Inner Product Cool Notation)

                                                                        nvv 1

                                                                        n

                                                                        iii vax

                                                                        1

                                                                        ii vxa

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis (cont)

                                                                        bull Spectral Representation

                                                                        where

                                                                        Check

                                                                        nvv 1

                                                                        n

                                                                        iii vax

                                                                        1

                                                                        ii vxa

                                                                        iii

                                                                        n

                                                                        iii

                                                                        n

                                                                        iiii avvavvavx

                                                                        1

                                                                        1

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis (cont)

                                                                        bull Spectral Representation

                                                                        where

                                                                        Check

                                                                        bull Matrix Notation where ie

                                                                        For the Basis Matrix

                                                                        nvv 1

                                                                        n

                                                                        iii vax

                                                                        1

                                                                        ii vxa

                                                                        iii

                                                                        n

                                                                        iii

                                                                        n

                                                                        iiii avvavvavx

                                                                        1

                                                                        1

                                                                        aBx Bxa tt xBa t

                                                                        nvvB 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Orthonormal Basis (cont)

                                                                        bull Spectral Representation

                                                                        where

                                                                        Check

                                                                        bull Matrix Notation where ie

                                                                        is called transform of

                                                                        (eg Fourier or Wavelet)

                                                                        nvv 1

                                                                        n

                                                                        iii vax

                                                                        1

                                                                        ii vxa

                                                                        iii

                                                                        n

                                                                        iii

                                                                        n

                                                                        iiii avvavvavx

                                                                        1

                                                                        1

                                                                        aBx Bxa tt xBa t

                                                                        xa

                                                                        Review of Linear Algebra (Cont)

                                                                        Parseval identity for

                                                                        in subsp genrsquod by o n basis

                                                                        x

                                                                        nvv 1

                                                                        2

                                                                        1

                                                                        22

                                                                        1

                                                                        2 aavxx

                                                                        n

                                                                        ii

                                                                        n

                                                                        ii

                                                                        Review of Linear Algebra (Cont)

                                                                        Parseval identity for

                                                                        in subsp genrsquod by o n basis

                                                                        bull Pythagorean theorem

                                                                        bull ldquoDecomposition of Energyrdquo

                                                                        bull ANOVA - sums of squares

                                                                        x

                                                                        nvv 1

                                                                        2

                                                                        1

                                                                        22

                                                                        1

                                                                        2 aavxx

                                                                        n

                                                                        ii

                                                                        n

                                                                        ii

                                                                        Review of Linear Algebra (Cont)

                                                                        Parseval identity for

                                                                        in subsp genrsquod by o n basis

                                                                        bull Pythagorean theorem

                                                                        bull ldquoDecomposition of Energyrdquo

                                                                        bull ANOVA - sums of squares

                                                                        bull Transform has same length as

                                                                        ie ldquorotation in rdquo

                                                                        x

                                                                        nvv 1

                                                                        2

                                                                        1

                                                                        22

                                                                        1

                                                                        2 aavxx

                                                                        n

                                                                        ii

                                                                        n

                                                                        ii

                                                                        a xd

                                                                        Projection of a Vector onto a Subspace

                                                                        bull Idea Member of that is Closest to

                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                        Review of Linear Algebra (Cont)x

                                                                        xV

                                                                        V

                                                                        Projection of a Vector onto a Subspace

                                                                        bull Idea Member of that is Closest to

                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                        bull Find that Solves

                                                                        (ldquoLeast Squaresrdquo)

                                                                        Review of Linear Algebra (Cont)x

                                                                        xV

                                                                        V

                                                                        VxPV vxVv

                                                                        min

                                                                        Projection of a Vector onto a Subspace

                                                                        bull Idea Member of that is Closest to

                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                        bull Find that Solves

                                                                        (ldquoLeast Squaresrdquo)

                                                                        bull For Inner Product (Hilbert) Space

                                                                        Exists and is Unique

                                                                        Review of Linear Algebra (Cont)x

                                                                        xV

                                                                        V

                                                                        VxPV vxVv

                                                                        min

                                                                        xPV

                                                                        Projection of a Vector onto a Subspace (cont)

                                                                        bull General Solution in for Basis Matrix

                                                                        Review of Linear Algebra (Cont)

                                                                        d VB

                                                                        xBBBBxP tVV

                                                                        tVVV

                                                                        1

                                                                        Projection of a Vector onto a Subspace (cont)

                                                                        bull General Solution in for Basis Matrix

                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                        (thus projection is another linear operation)

                                                                        Review of Linear Algebra (Cont)

                                                                        d VB

                                                                        xBBBBxP tVV

                                                                        tVVV

                                                                        1

                                                                        tVV

                                                                        tVVV BBBBP

                                                                        1

                                                                        Projection of a Vector onto a Subspace (cont)

                                                                        bull General Solution in for Basis Matrix

                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                        (thus projection is another linear operation)

                                                                        (note same operation underlies least squares)

                                                                        Review of Linear Algebra (Cont)

                                                                        d VB

                                                                        xBBBBxP tVV

                                                                        tVVV

                                                                        1

                                                                        tVV

                                                                        tVVV BBBBP

                                                                        1

                                                                        Review of Linear Algebra (Cont)

                                                                        Projection using Orthonormal Basis

                                                                        bull Basis Matrix is Orthonormal

                                                                        nnVtV IBB

                                                                        10

                                                                        01

                                                                        1

                                                                        111

                                                                        1

                                                                        1

                                                                        nnn

                                                                        n

                                                                        ntn

                                                                        t

                                                                        vvvv

                                                                        vvvv

                                                                        vv

                                                                        v

                                                                        v

                                                                        nvv 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Projection using Orthonormal Basis

                                                                        bull Basis Matrix is Orthonormal

                                                                        bull So =

                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                        (Recall Right Multrsquon)

                                                                        nnVtV IBB

                                                                        10

                                                                        01

                                                                        1

                                                                        111

                                                                        1

                                                                        1

                                                                        nnn

                                                                        n

                                                                        ntn

                                                                        t

                                                                        vvvv

                                                                        vvvv

                                                                        vv

                                                                        v

                                                                        v

                                                                        xBBxP tVVV

                                                                        x V

                                                                        nvv 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Projection using Orthonormal Basis (cont)

                                                                        bull For Orthogonal Complement

                                                                        and

                                                                        V

                                                                        xPxPx VV 222xPxPx VV

                                                                        Review of Linear Algebra (Cont)

                                                                        Projection using Orthonormal Basis (cont)

                                                                        bull For Orthogonal Complement

                                                                        and

                                                                        bull Parseval Inequality

                                                                        V

                                                                        xPxPx VV 222xPxPx VV

                                                                        2

                                                                        1

                                                                        22

                                                                        1

                                                                        22 aavxxxP

                                                                        n

                                                                        ii

                                                                        n

                                                                        iiV

                                                                        Review of Linear Algebra (Cont)

                                                                        (Real) Unitary Matrices with

                                                                        bull Orthonormal Basis Matrix

                                                                        (So All of Above Applies)

                                                                        ddU IUU t

                                                                        Review of Linear Algebra (Cont)

                                                                        (Real) Unitary Matrices with

                                                                        bull Orthonormal Basis Matrix

                                                                        (So All of Above Applies)

                                                                        bull Note Transformrsquon is Distance Preserving

                                                                        ddU IUU t

                                                                        yxdyxyxyxUyUxUdn

                                                                        i ii 2

                                                                        1

                                                                        Review of Linear Algebra (Cont)

                                                                        (Real) Unitary Matrices with

                                                                        bull Orthonormal Basis Matrix

                                                                        (So All of Above Applies)

                                                                        bull Note Transformrsquon is Distance Preserving

                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                        ddU IUU t

                                                                        yxdyxyxyxUyUxUdn

                                                                        i ii 2

                                                                        1

                                                                        U

                                                                        Review of Linear Algebra (Cont)

                                                                        Singular Value Decomposition (SVD)

                                                                        For a Matrix

                                                                        Find

                                                                        ndX

                                                                        Review of Linear Algebra (Cont)

                                                                        Singular Value Decomposition (SVD)

                                                                        For a Matrix

                                                                        Find a Diagonal Matrix

                                                                        with Entries

                                                                        called Singular Values

                                                                        ndX

                                                                        ndS

                                                                        )min(1 ndss

                                                                        Review of Linear Algebra (Cont)

                                                                        Singular Value Decomposition (SVD)

                                                                        For a Matrix

                                                                        Find a Diagonal Matrix

                                                                        with Entries

                                                                        called Singular Values

                                                                        And Unitary (Rotation) Matrices

                                                                        (recall )

                                                                        ndX

                                                                        ndS

                                                                        )min(1 ndss

                                                                        ddU nnV

                                                                        IVVUU tt

                                                                        Review of Linear Algebra (Cont)

                                                                        Singular Value Decomposition (SVD)

                                                                        For a Matrix

                                                                        Find a Diagonal Matrix

                                                                        with Entries

                                                                        called Singular Values

                                                                        And Unitary (Rotation) Matrices

                                                                        (recall )

                                                                        So That

                                                                        ndX

                                                                        ndS

                                                                        )min(1 ndss

                                                                        ddU nnV

                                                                        IVVUU tt tUSVX

                                                                        Review of Linear Algebra (Cont)

                                                                        Intuition behind Singular Value Decomposition

                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                        vVSUvVSUvX tt

                                                                        Review of Linear Algebra (Cont)

                                                                        Intuition behind Singular Value Decomposition

                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                        bull First rotate

                                                                        X

                                                                        vVSUvVSUvX tt

                                                                        Review of Linear Algebra (Cont)

                                                                        Intuition behind Singular Value Decomposition

                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                        bull First rotate

                                                                        bull Second rescale coordinate axes (by )

                                                                        X

                                                                        vVSUvVSUvX tt

                                                                        is

                                                                        Review of Linear Algebra (Cont)

                                                                        Intuition behind Singular Value Decomposition

                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                        bull First rotate

                                                                        bull Second rescale coordinate axes (by )

                                                                        bull Third rotate again

                                                                        X

                                                                        vVSUvVSUvX tt

                                                                        is

                                                                        Review of Linear Algebra (Cont)

                                                                        Intuition behind Singular Value Decomposition

                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                        bull First rotate

                                                                        bull Second rescale coordinate axes (by )

                                                                        bull Third rotate again

                                                                        bull ie have diagonalized the transformation

                                                                        X

                                                                        vVSUvVSUvX tt

                                                                        is

                                                                        SVD Compact Representation

                                                                        Useful Labeling

                                                                        Singular Values in Increasing Order

                                                                        Review of Linear Algebra (Cont)

                                                                        )min(1 dnss

                                                                        SVD Compact Representation

                                                                        Useful Labeling

                                                                        Singular Values in Increasing Order

                                                                        Note singular values = 0 can be omitted

                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                        Review of Linear Algebra (Cont)

                                                                        )min(1 dnss

                                                                        Review of Linear Algebra (Cont)

                                                                        r

                                                                        SVD Compact Representation

                                                                        Useful Labeling

                                                                        Singular Values in Increasing Order

                                                                        Note singular values = 0 can be omitted

                                                                        Let = of positive singular values

                                                                        )min(1 dnss

                                                                        Review of Linear Algebra (Cont)

                                                                        r

                                                                        SVD Compact Representation

                                                                        Useful Labeling

                                                                        Singular Values in Increasing Order

                                                                        Note singular values = 0 can be omitted

                                                                        Let = of positive singular values

                                                                        Then

                                                                        Where are truncations of

                                                                        trnrrrd VSUX

                                                                        VSU

                                                                        )min(1 dnss

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Full Representation

                                                                        =

                                                                        Graphics Display Assumes

                                                                        ndX ddU ndS nn

                                                                        tV

                                                                        nd

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Full Representation

                                                                        =

                                                                        Full Rank Basis Matrix

                                                                        ndX ddU ndS nn

                                                                        tV

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Full Representation

                                                                        =

                                                                        Full Rank Basis Matrix

                                                                        All 0s in Bottom

                                                                        ndX ddU ndS nn

                                                                        tV

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Reduced Representation

                                                                        =

                                                                        These Columns Get 0ed Out

                                                                        ndX ddU nnS nn

                                                                        tV

                                                                        nnd 0

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Reduced Representation

                                                                        =ndX ndU nnS nn

                                                                        tV

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Reduced Representation

                                                                        =

                                                                        Also Some of These May be 0

                                                                        ndX ndU nnS nn

                                                                        tV

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Compact Representation

                                                                        = ndX rdU

                                                                        rrS nrtV

                                                                        0

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Compact Representation

                                                                        =

                                                                        These Get 0ed Out

                                                                        ndX rdU

                                                                        rrS nrtV

                                                                        0

                                                                        Review of Linear Algebra (Cont)

                                                                        SVD Compact Representation

                                                                        = ndX rdU

                                                                        rrS nrtV

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition

                                                                        For a (Symmetric) Square Matrix

                                                                        Find

                                                                        ddX

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition

                                                                        For a (Symmetric) Square Matrix

                                                                        Find a Diagonal Matrix

                                                                        Called Eigenvalues

                                                                        Convenient Ordering

                                                                        ddX

                                                                        d

                                                                        D

                                                                        0

                                                                        01

                                                                        n 1

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition

                                                                        For a (Symmetric) Square Matrix

                                                                        Find a Diagonal Matrix

                                                                        And an Orthonormal Matrix

                                                                        (ie )

                                                                        ddX

                                                                        d

                                                                        D

                                                                        0

                                                                        01

                                                                        ddB

                                                                        ddtt IBBBB

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition

                                                                        For a (Symmetric) Square Matrix

                                                                        Find a Diagonal Matrix

                                                                        And an Orthonormal Matrix

                                                                        (ie )

                                                                        So that ie

                                                                        ddX

                                                                        d

                                                                        D

                                                                        0

                                                                        01

                                                                        ddB

                                                                        ddtt IBBBB

                                                                        DBBX tBDBX

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                        (looks similar)

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                        Complex (uses )

                                                                        VU

                                                                        1i

                                                                        Review of Linear Algebra (Cont)

                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                        VU

                                                                        X

                                                                        BVU

                                                                        1i

                                                                        • Participant Presentations
                                                                        • PCA to find clusters
                                                                        • Statistical Smoothing
                                                                        • Kernel Density Estimation
                                                                        • Scatterplot Smoothing
                                                                        • Statistical Smoothing (2)
                                                                        • SiZer Background
                                                                        • SiZer Background (2)
                                                                        • SiZer Background (3)
                                                                        • SiZer Background (4)
                                                                        • SiZer Background (5)
                                                                        • SiZer Overview
                                                                        • PCA to find clusters (2)
                                                                        • PCA to find clusters (3)
                                                                        • PCA to find clusters (4)
                                                                        • PCA to find clusters (5)
                                                                        • PCA to find clusters (6)
                                                                        • PCA to find clusters (7)
                                                                        • Recall Yeast Cell Cycle Data
                                                                        • Yeast Cell Cycle Data FDA View
                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                        • Frequency 2 Analysis
                                                                        • Frequency 2 Analysis (2)
                                                                        • Frequency 2 Analysis (3)
                                                                        • Yeast Cell Cycle
                                                                        • SiZer Study of Distrsquon of Angles
                                                                        • Reclassification of Major Genes
                                                                        • Compare to Previous Classifrsquon
                                                                        • New Subpopulation View
                                                                        • New Subpopulation View (2)
                                                                        • Detailed Look at PCA
                                                                        • PCA Rediscovery ndash Renaming
                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                        • An Interesting Historical Note
                                                                        • An Interesting Historical Note (2)
                                                                        • Detailed Look at PCA (2)
                                                                        • Detailed Look at PCA (3)
                                                                        • Review of Linear Algebra
                                                                        • Review of Linear Algebra (2)
                                                                        • Review of Linear Algebra (3)
                                                                        • Review of Linear Algebra (Cont)
                                                                        • Review of Linear Algebra (Cont) (2)
                                                                        • Review of Linear Algebra (Cont) (3)
                                                                        • Review of Linear Algebra (Cont) (4)
                                                                        • Review of Linear Algebra (Cont) (5)
                                                                        • Review of Linear Algebra (Cont) (6)
                                                                        • Review of Linear Algebra (Cont) (7)
                                                                        • Review of Linear Algebra (Cont) (8)
                                                                        • Review of Linear Algebra (Cont) (9)
                                                                        • Review of Linear Algebra (Cont) (10)
                                                                        • Review of Linear Algebra (Cont) (11)
                                                                        • Review of Linear Algebra (Cont) (12)
                                                                        • Review of Linear Algebra (Cont) (13)
                                                                        • Review of Linear Algebra (Cont) (14)
                                                                        • Review of Linear Algebra (Cont) (15)
                                                                        • Review of Linear Algebra (Cont) (16)
                                                                        • Review of Linear Algebra (Cont) (17)
                                                                        • Review of Linear Algebra (Cont) (18)
                                                                        • Review of Linear Algebra (Cont) (19)
                                                                        • Review of Linear Algebra (Cont) (20)
                                                                        • Review of Linear Algebra (Cont) (21)
                                                                        • Review of Linear Algebra (Cont) (22)
                                                                        • Review of Linear Algebra (Cont) (23)
                                                                        • Review of Linear Algebra (Cont) (24)
                                                                        • Review of Linear Algebra (Cont) (25)
                                                                        • Review of Linear Algebra (Cont) (26)
                                                                        • Review of Linear Algebra (Cont) (27)
                                                                        • Review of Linear Algebra (Cont) (28)
                                                                        • Review of Linear Algebra (Cont) (29)
                                                                        • Review of Linear Algebra (Cont) (30)
                                                                        • Review of Linear Algebra (Cont) (31)
                                                                        • Review of Linear Algebra (Cont) (32)
                                                                        • Review of Linear Algebra (Cont) (33)
                                                                        • Review of Linear Algebra (Cont) (34)
                                                                        • Review of Linear Algebra (Cont) (35)
                                                                        • Review of Linear Algebra (Cont) (36)
                                                                        • Review of Linear Algebra (Cont) (37)
                                                                        • Review of Linear Algebra (Cont) (38)
                                                                        • Review of Linear Algebra (Cont) (39)
                                                                        • Review of Linear Algebra (Cont) (40)
                                                                        • Review of Linear Algebra (Cont) (41)
                                                                        • Review of Linear Algebra (Cont) (42)
                                                                        • Review of Linear Algebra (Cont) (43)
                                                                        • Review of Linear Algebra (Cont) (44)
                                                                        • Review of Linear Algebra (Cont) (45)
                                                                        • Review of Linear Algebra (Cont) (46)
                                                                        • Review of Linear Algebra (Cont) (47)
                                                                        • Review of Linear Algebra (Cont) (48)
                                                                        • Review of Linear Algebra (Cont) (49)
                                                                        • Review of Linear Algebra (Cont) (50)
                                                                        • Review of Linear Algebra (Cont) (51)
                                                                        • Review of Linear Algebra (Cont) (52)
                                                                        • Review of Linear Algebra (Cont) (53)
                                                                        • Review of Linear Algebra (Cont) (54)
                                                                        • Review of Linear Algebra (Cont) (55)
                                                                        • Review of Linear Algebra (Cont) (56)
                                                                        • Review of Linear Algebra (Cont) (57)
                                                                        • Review of Linear Algebra (Cont) (58)
                                                                        • Review of Linear Algebra (Cont) (59)
                                                                        • Review of Linear Algebra (Cont) (60)
                                                                        • Review of Linear Algebra (Cont) (61)
                                                                        • Review of Linear Algebra (Cont) (62)
                                                                        • Review of Linear Algebra (Cont) (63)
                                                                        • Review of Linear Algebra (Cont) (64)
                                                                        • Review of Linear Algebra (Cont) (65)
                                                                        • Review of Linear Algebra (Cont) (66)
                                                                        • Review of Linear Algebra (Cont) (67)
                                                                        • Review of Linear Algebra (Cont) (68)
                                                                        • Review of Linear Algebra (Cont) (69)
                                                                        • Review of Linear Algebra (Cont) (70)
                                                                        • Review of Linear Algebra (Cont) (71)
                                                                        • Review of Linear Algebra (Cont) (72)
                                                                        • Review of Linear Algebra (Cont) (73)
                                                                        • Review of Linear Algebra (Cont) (74)
                                                                        • Review of Linear Algebra (Cont) (75)
                                                                        • Review of Linear Algebra (Cont) (76)
                                                                        • Review of Linear Algebra (Cont) (77)
                                                                        • Review of Linear Algebra (Cont) (78)
                                                                        • Review of Linear Algebra (Cont) (79)
                                                                        • Review of Linear Algebra (Cont) (80)
                                                                        • Review of Linear Algebra (Cont) (81)

                                                                          PCA Rediscovery ndash Renaming

                                                                          Statistics Principal Component Analysis (PCA)

                                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                          PCA Rediscovery ndash Renaming

                                                                          Statistics Principal Component Analysis (PCA)

                                                                          Social Sciences Factor Analysis (PCA is a subset)

                                                                          Probability Electrical EngKarhunen ndash Loeve expansion

                                                                          Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                          Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                          An Interesting Historical Note

                                                                          The 1st () application of PCA to Functional

                                                                          Data Analysis

                                                                          An Interesting Historical Note

                                                                          The 1st () application of PCA to Functional

                                                                          Data Analysis

                                                                          Rao (1958)

                                                                          1st Paper with ldquoCurves as Data Objectsrdquo

                                                                          viewpoint

                                                                          Detailed Look at PCA

                                                                          Three Important (amp Interesting) Viewpoints

                                                                          1 Mathematics

                                                                          2 Numerics

                                                                          3 Statistics

                                                                          Goal Study Interrelationships

                                                                          Detailed Look at PCA

                                                                          Three Important (amp Interesting) Viewpoints

                                                                          1 Mathematics

                                                                          2 Numerics

                                                                          3 Statistics

                                                                          1st Review Linear Alg and Multivar Prob

                                                                          Review of Linear Algebra

                                                                          Vector Space

                                                                          bull set of ldquovectorsrdquo

                                                                          bull and ldquoscalarsrdquo (coefficients)

                                                                          xa

                                                                          Review of Linear Algebra

                                                                          Vector Space

                                                                          bull set of ldquovectorsrdquo

                                                                          bull and ldquoscalarsrdquo (coefficients)

                                                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                          ( in space)

                                                                          xa

                                                                          i

                                                                          ii xa

                                                                          Review of Linear Algebra

                                                                          Vector Space

                                                                          bull set of ldquovectorsrdquo

                                                                          bull and ldquoscalarsrdquo (coefficients)

                                                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                          ( in space)

                                                                          eg

                                                                          ldquo dim Euclidrsquon spacerdquo

                                                                          xa

                                                                          i

                                                                          ii xa

                                                                          d

                                                                          d

                                                                          d xx

                                                                          x

                                                                          x

                                                                          x 1

                                                                          1

                                                                          d

                                                                          Review of Linear Algebra (Cont)

                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                          Review of Linear Algebra (Cont)

                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                          Review of Linear Algebra (Cont)

                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                          Note Planes not Through the Origin

                                                                          are not Subspaces

                                                                          (Do not Contain )00 x

                                                                          Review of Linear Algebra (Cont)

                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                          (all Linear Combos of them =

                                                                          = Containing Hyperplane

                                                                          through Origin)

                                                                          Review of Linear Algebra (Cont)

                                                                          Basis of Subspace Set of Vectors that

                                                                          bull Span ie Everything is a Lin Com of them

                                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                          Review of Linear Algebra (Cont)

                                                                          Basis of Subspace Set of Vectors that

                                                                          bull Span ie Everything is a Lin Com of them

                                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                          bull eg ldquoUnit Vector Basisrdquo

                                                                          bull Since

                                                                          d

                                                                          1

                                                                          0

                                                                          0

                                                                          0

                                                                          1

                                                                          0

                                                                          0

                                                                          0

                                                                          1

                                                                          1

                                                                          0

                                                                          0

                                                                          0

                                                                          1

                                                                          0

                                                                          0

                                                                          0

                                                                          1

                                                                          212

                                                                          1

                                                                          d

                                                                          d

                                                                          xxx

                                                                          x

                                                                          x

                                                                          x

                                                                          Review of Linear Algebra (Cont)

                                                                          Basis Matrix of subspace of

                                                                          Given a basis

                                                                          create matrix of columns

                                                                          dnvv 1

                                                                          nddnd

                                                                          n

                                                                          n

                                                                          vv

                                                                          vv

                                                                          vvB

                                                                          1

                                                                          111

                                                                          1

                                                                          Review of Linear Algebra (Cont)

                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                          where

                                                                          n

                                                                          iii aBva

                                                                          1

                                                                          na

                                                                          a

                                                                          a 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                          where

                                                                          Note Right Multiplication Gives

                                                                          Linear Combination of Column Vectors

                                                                          n

                                                                          iii aBva

                                                                          1

                                                                          na

                                                                          a

                                                                          a 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                          where

                                                                          Check sizes

                                                                          n

                                                                          iii aBva

                                                                          1

                                                                          na

                                                                          a

                                                                          a 1

                                                                          )1()(1 nndd

                                                                          Review of Linear Algebra (Cont)

                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                          For matrices

                                                                          Define the Matrix Product

                                                                          mkk

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          nmm

                                                                          n

                                                                          bb

                                                                          bb

                                                                          B

                                                                          1

                                                                          111

                                                                          m

                                                                          iniik

                                                                          m

                                                                          iiik

                                                                          m

                                                                          inii

                                                                          m

                                                                          iii

                                                                          baba

                                                                          baba

                                                                          AB

                                                                          1

                                                                          11

                                                                          11

                                                                          111

                                                                          Review of Linear Algebra (Cont)

                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                          For matrices

                                                                          Define the Matrix Product

                                                                          (Inner Products of Rows With Columns )

                                                                          mkk

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          nmm

                                                                          n

                                                                          bb

                                                                          bb

                                                                          B

                                                                          1

                                                                          111

                                                                          m

                                                                          iniik

                                                                          m

                                                                          iiik

                                                                          m

                                                                          inii

                                                                          m

                                                                          iii

                                                                          baba

                                                                          baba

                                                                          AB

                                                                          1

                                                                          11

                                                                          11

                                                                          111

                                                                          A B

                                                                          Review of Linear Algebra (Cont)

                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                          For matrices

                                                                          Define the Matrix Product

                                                                          (Inner Products of Rows With Columns )

                                                                          (Composition of Linear Transformations)

                                                                          mkk

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          nmm

                                                                          n

                                                                          bb

                                                                          bb

                                                                          B

                                                                          1

                                                                          111

                                                                          m

                                                                          iniik

                                                                          m

                                                                          iiik

                                                                          m

                                                                          inii

                                                                          m

                                                                          iii

                                                                          baba

                                                                          baba

                                                                          AB

                                                                          1

                                                                          11

                                                                          11

                                                                          111

                                                                          A B

                                                                          Review of Linear Algebra (Cont)

                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                          For matrices

                                                                          Define the Matrix Product

                                                                          (Inner Products of Rows With Columns )

                                                                          (Composition of Linear Transformations)

                                                                          Often Useful to Check Sizes

                                                                          mkk

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          nmm

                                                                          n

                                                                          bb

                                                                          bb

                                                                          B

                                                                          1

                                                                          111

                                                                          m

                                                                          iniik

                                                                          m

                                                                          iiik

                                                                          m

                                                                          inii

                                                                          m

                                                                          iii

                                                                          baba

                                                                          baba

                                                                          AB

                                                                          1

                                                                          11

                                                                          11

                                                                          111

                                                                          nmmknk

                                                                          A B

                                                                          Review of Linear Algebra (Cont)

                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                          For matrices

                                                                          Define the Matrix Product

                                                                          (Inner Products of Rows With Columns )

                                                                          (Composition of Linear Transformations)

                                                                          Often Useful to Check Sizes

                                                                          mkk

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          nmm

                                                                          n

                                                                          bb

                                                                          bb

                                                                          B

                                                                          1

                                                                          111

                                                                          m

                                                                          iniik

                                                                          m

                                                                          iiik

                                                                          m

                                                                          inii

                                                                          m

                                                                          iii

                                                                          baba

                                                                          baba

                                                                          AB

                                                                          1

                                                                          11

                                                                          11

                                                                          111

                                                                          nmmknk

                                                                          A B

                                                                          Review of Linear Algebra (Cont)

                                                                          Matrix Trace

                                                                          bull For a Square Matrix

                                                                          bull Define

                                                                          mmm

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          m

                                                                          iiiaAtr

                                                                          1)(

                                                                          Review of Linear Algebra (Cont)

                                                                          Matrix Trace

                                                                          bull For a Square Matrix

                                                                          bull Define

                                                                          bull Trace Commutes with Matrix Multiplication

                                                                          mmm

                                                                          m

                                                                          aa

                                                                          aa

                                                                          A

                                                                          1

                                                                          111

                                                                          m

                                                                          iiiaAtr

                                                                          1)(

                                                                          BAtrABtr

                                                                          Review of Linear Algebra (Cont)

                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                          bull Number of Elements in a Basis (Unique)

                                                                          Review of Linear Algebra (Cont)

                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                          bull Number of Elements in a Basis (Unique)

                                                                          bull (Use Basis Above)

                                                                          bull eg dim of a line is 1

                                                                          bull eg dim of a plane is 2

                                                                          dd dim

                                                                          Review of Linear Algebra (Cont)

                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                          bull Number of Elements in a Basis (Unique)

                                                                          bull (Use Basis Above)

                                                                          bull eg dim of a line is 1

                                                                          bull eg dim of a plane is 2

                                                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                                                          (in Statistical Uses eg ANOVA)

                                                                          dd dim

                                                                          Review of Linear Algebra (Cont)

                                                                          Norm of a Vector

                                                                          bull in d 21

                                                                          21

                                                                          1

                                                                          2 xxxx td

                                                                          jj

                                                                          Review of Linear Algebra (Cont)

                                                                          Norm of a Vector

                                                                          bull in

                                                                          bull Idea length of the vector

                                                                          d 2121

                                                                          1

                                                                          2 xxxx td

                                                                          jj

                                                                          Review of Linear Algebra (Cont)

                                                                          Norm of a Vector

                                                                          bull in

                                                                          bull Idea length of the vector

                                                                          bull Note strange properties for high

                                                                          eg ldquolength of diagonal of unit cuberdquo =

                                                                          d 2121

                                                                          1

                                                                          2 xxxx td

                                                                          jj

                                                                          d

                                                                          d

                                                                          Review of Linear Algebra (Cont)

                                                                          Norm of a Vector (cont)

                                                                          bull Length Normalized Vector

                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                          amp is a Direction Vector)

                                                                          x

                                                                          x

                                                                          Review of Linear Algebra (Cont)

                                                                          Norm of a Vector (cont)

                                                                          bull Length Normalized Vector

                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                          amp is a Direction Vector)

                                                                          bull Define Distance as

                                                                          x

                                                                          x

                                                                          yxyxyxyxd t

                                                                          Review of Linear Algebra (Cont)

                                                                          Inner (Dot Scalar) Product

                                                                          bull for Vectors and

                                                                          yxyxyx td

                                                                          jjj

                                                                          1

                                                                          x y

                                                                          Review of Linear Algebra (Cont)

                                                                          Inner (Dot Scalar) Product

                                                                          bull for Vectors and

                                                                          bull Related to Norm via

                                                                          yxyxyx td

                                                                          jjj

                                                                          1

                                                                          x y

                                                                          21 xxx

                                                                          Review of Linear Algebra (Cont)

                                                                          Inner (Dot Scalar) Product (cont)

                                                                          bull measures ldquoangle between and rdquo as

                                                                          yyxx

                                                                          yx

                                                                          yx

                                                                          yxyxangle

                                                                          tt

                                                                          t

                                                                          11 cos

                                                                          cos

                                                                          x y

                                                                          Review of Linear Algebra (Cont)

                                                                          Inner (Dot Scalar) Product (cont)

                                                                          bull measures ldquoangle between and rdquo as

                                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                                          if and only if

                                                                          yyxx

                                                                          yx

                                                                          yx

                                                                          yxyxangle

                                                                          tt

                                                                          t

                                                                          11 cos

                                                                          cos

                                                                          x y

                                                                          yx 0 yx

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis

                                                                          bull All Orthogonal to each other

                                                                          ie for

                                                                          bull All have Length 1

                                                                          ie for

                                                                          nvv 1

                                                                          1 ii vv

                                                                          0 ii vv ii

                                                                          ni 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis (cont)

                                                                          bull Spectral Representation

                                                                          where

                                                                          nvv 1

                                                                          n

                                                                          iii vax

                                                                          1

                                                                          ii vxa

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis (cont)

                                                                          bull Spectral Representation

                                                                          where

                                                                          (Coefficient is Inner Product Cool Notation)

                                                                          nvv 1

                                                                          n

                                                                          iii vax

                                                                          1

                                                                          ii vxa

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis (cont)

                                                                          bull Spectral Representation

                                                                          where

                                                                          Check

                                                                          nvv 1

                                                                          n

                                                                          iii vax

                                                                          1

                                                                          ii vxa

                                                                          iii

                                                                          n

                                                                          iii

                                                                          n

                                                                          iiii avvavvavx

                                                                          1

                                                                          1

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis (cont)

                                                                          bull Spectral Representation

                                                                          where

                                                                          Check

                                                                          bull Matrix Notation where ie

                                                                          For the Basis Matrix

                                                                          nvv 1

                                                                          n

                                                                          iii vax

                                                                          1

                                                                          ii vxa

                                                                          iii

                                                                          n

                                                                          iii

                                                                          n

                                                                          iiii avvavvavx

                                                                          1

                                                                          1

                                                                          aBx Bxa tt xBa t

                                                                          nvvB 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Orthonormal Basis (cont)

                                                                          bull Spectral Representation

                                                                          where

                                                                          Check

                                                                          bull Matrix Notation where ie

                                                                          is called transform of

                                                                          (eg Fourier or Wavelet)

                                                                          nvv 1

                                                                          n

                                                                          iii vax

                                                                          1

                                                                          ii vxa

                                                                          iii

                                                                          n

                                                                          iii

                                                                          n

                                                                          iiii avvavvavx

                                                                          1

                                                                          1

                                                                          aBx Bxa tt xBa t

                                                                          xa

                                                                          Review of Linear Algebra (Cont)

                                                                          Parseval identity for

                                                                          in subsp genrsquod by o n basis

                                                                          x

                                                                          nvv 1

                                                                          2

                                                                          1

                                                                          22

                                                                          1

                                                                          2 aavxx

                                                                          n

                                                                          ii

                                                                          n

                                                                          ii

                                                                          Review of Linear Algebra (Cont)

                                                                          Parseval identity for

                                                                          in subsp genrsquod by o n basis

                                                                          bull Pythagorean theorem

                                                                          bull ldquoDecomposition of Energyrdquo

                                                                          bull ANOVA - sums of squares

                                                                          x

                                                                          nvv 1

                                                                          2

                                                                          1

                                                                          22

                                                                          1

                                                                          2 aavxx

                                                                          n

                                                                          ii

                                                                          n

                                                                          ii

                                                                          Review of Linear Algebra (Cont)

                                                                          Parseval identity for

                                                                          in subsp genrsquod by o n basis

                                                                          bull Pythagorean theorem

                                                                          bull ldquoDecomposition of Energyrdquo

                                                                          bull ANOVA - sums of squares

                                                                          bull Transform has same length as

                                                                          ie ldquorotation in rdquo

                                                                          x

                                                                          nvv 1

                                                                          2

                                                                          1

                                                                          22

                                                                          1

                                                                          2 aavxx

                                                                          n

                                                                          ii

                                                                          n

                                                                          ii

                                                                          a xd

                                                                          Projection of a Vector onto a Subspace

                                                                          bull Idea Member of that is Closest to

                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                          Review of Linear Algebra (Cont)x

                                                                          xV

                                                                          V

                                                                          Projection of a Vector onto a Subspace

                                                                          bull Idea Member of that is Closest to

                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                          bull Find that Solves

                                                                          (ldquoLeast Squaresrdquo)

                                                                          Review of Linear Algebra (Cont)x

                                                                          xV

                                                                          V

                                                                          VxPV vxVv

                                                                          min

                                                                          Projection of a Vector onto a Subspace

                                                                          bull Idea Member of that is Closest to

                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                          bull Find that Solves

                                                                          (ldquoLeast Squaresrdquo)

                                                                          bull For Inner Product (Hilbert) Space

                                                                          Exists and is Unique

                                                                          Review of Linear Algebra (Cont)x

                                                                          xV

                                                                          V

                                                                          VxPV vxVv

                                                                          min

                                                                          xPV

                                                                          Projection of a Vector onto a Subspace (cont)

                                                                          bull General Solution in for Basis Matrix

                                                                          Review of Linear Algebra (Cont)

                                                                          d VB

                                                                          xBBBBxP tVV

                                                                          tVVV

                                                                          1

                                                                          Projection of a Vector onto a Subspace (cont)

                                                                          bull General Solution in for Basis Matrix

                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                          (thus projection is another linear operation)

                                                                          Review of Linear Algebra (Cont)

                                                                          d VB

                                                                          xBBBBxP tVV

                                                                          tVVV

                                                                          1

                                                                          tVV

                                                                          tVVV BBBBP

                                                                          1

                                                                          Projection of a Vector onto a Subspace (cont)

                                                                          bull General Solution in for Basis Matrix

                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                          (thus projection is another linear operation)

                                                                          (note same operation underlies least squares)

                                                                          Review of Linear Algebra (Cont)

                                                                          d VB

                                                                          xBBBBxP tVV

                                                                          tVVV

                                                                          1

                                                                          tVV

                                                                          tVVV BBBBP

                                                                          1

                                                                          Review of Linear Algebra (Cont)

                                                                          Projection using Orthonormal Basis

                                                                          bull Basis Matrix is Orthonormal

                                                                          nnVtV IBB

                                                                          10

                                                                          01

                                                                          1

                                                                          111

                                                                          1

                                                                          1

                                                                          nnn

                                                                          n

                                                                          ntn

                                                                          t

                                                                          vvvv

                                                                          vvvv

                                                                          vv

                                                                          v

                                                                          v

                                                                          nvv 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Projection using Orthonormal Basis

                                                                          bull Basis Matrix is Orthonormal

                                                                          bull So =

                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                          (Recall Right Multrsquon)

                                                                          nnVtV IBB

                                                                          10

                                                                          01

                                                                          1

                                                                          111

                                                                          1

                                                                          1

                                                                          nnn

                                                                          n

                                                                          ntn

                                                                          t

                                                                          vvvv

                                                                          vvvv

                                                                          vv

                                                                          v

                                                                          v

                                                                          xBBxP tVVV

                                                                          x V

                                                                          nvv 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Projection using Orthonormal Basis (cont)

                                                                          bull For Orthogonal Complement

                                                                          and

                                                                          V

                                                                          xPxPx VV 222xPxPx VV

                                                                          Review of Linear Algebra (Cont)

                                                                          Projection using Orthonormal Basis (cont)

                                                                          bull For Orthogonal Complement

                                                                          and

                                                                          bull Parseval Inequality

                                                                          V

                                                                          xPxPx VV 222xPxPx VV

                                                                          2

                                                                          1

                                                                          22

                                                                          1

                                                                          22 aavxxxP

                                                                          n

                                                                          ii

                                                                          n

                                                                          iiV

                                                                          Review of Linear Algebra (Cont)

                                                                          (Real) Unitary Matrices with

                                                                          bull Orthonormal Basis Matrix

                                                                          (So All of Above Applies)

                                                                          ddU IUU t

                                                                          Review of Linear Algebra (Cont)

                                                                          (Real) Unitary Matrices with

                                                                          bull Orthonormal Basis Matrix

                                                                          (So All of Above Applies)

                                                                          bull Note Transformrsquon is Distance Preserving

                                                                          ddU IUU t

                                                                          yxdyxyxyxUyUxUdn

                                                                          i ii 2

                                                                          1

                                                                          Review of Linear Algebra (Cont)

                                                                          (Real) Unitary Matrices with

                                                                          bull Orthonormal Basis Matrix

                                                                          (So All of Above Applies)

                                                                          bull Note Transformrsquon is Distance Preserving

                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                          ddU IUU t

                                                                          yxdyxyxyxUyUxUdn

                                                                          i ii 2

                                                                          1

                                                                          U

                                                                          Review of Linear Algebra (Cont)

                                                                          Singular Value Decomposition (SVD)

                                                                          For a Matrix

                                                                          Find

                                                                          ndX

                                                                          Review of Linear Algebra (Cont)

                                                                          Singular Value Decomposition (SVD)

                                                                          For a Matrix

                                                                          Find a Diagonal Matrix

                                                                          with Entries

                                                                          called Singular Values

                                                                          ndX

                                                                          ndS

                                                                          )min(1 ndss

                                                                          Review of Linear Algebra (Cont)

                                                                          Singular Value Decomposition (SVD)

                                                                          For a Matrix

                                                                          Find a Diagonal Matrix

                                                                          with Entries

                                                                          called Singular Values

                                                                          And Unitary (Rotation) Matrices

                                                                          (recall )

                                                                          ndX

                                                                          ndS

                                                                          )min(1 ndss

                                                                          ddU nnV

                                                                          IVVUU tt

                                                                          Review of Linear Algebra (Cont)

                                                                          Singular Value Decomposition (SVD)

                                                                          For a Matrix

                                                                          Find a Diagonal Matrix

                                                                          with Entries

                                                                          called Singular Values

                                                                          And Unitary (Rotation) Matrices

                                                                          (recall )

                                                                          So That

                                                                          ndX

                                                                          ndS

                                                                          )min(1 ndss

                                                                          ddU nnV

                                                                          IVVUU tt tUSVX

                                                                          Review of Linear Algebra (Cont)

                                                                          Intuition behind Singular Value Decomposition

                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                          vVSUvVSUvX tt

                                                                          Review of Linear Algebra (Cont)

                                                                          Intuition behind Singular Value Decomposition

                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                          bull First rotate

                                                                          X

                                                                          vVSUvVSUvX tt

                                                                          Review of Linear Algebra (Cont)

                                                                          Intuition behind Singular Value Decomposition

                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                          bull First rotate

                                                                          bull Second rescale coordinate axes (by )

                                                                          X

                                                                          vVSUvVSUvX tt

                                                                          is

                                                                          Review of Linear Algebra (Cont)

                                                                          Intuition behind Singular Value Decomposition

                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                          bull First rotate

                                                                          bull Second rescale coordinate axes (by )

                                                                          bull Third rotate again

                                                                          X

                                                                          vVSUvVSUvX tt

                                                                          is

                                                                          Review of Linear Algebra (Cont)

                                                                          Intuition behind Singular Value Decomposition

                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                          bull First rotate

                                                                          bull Second rescale coordinate axes (by )

                                                                          bull Third rotate again

                                                                          bull ie have diagonalized the transformation

                                                                          X

                                                                          vVSUvVSUvX tt

                                                                          is

                                                                          SVD Compact Representation

                                                                          Useful Labeling

                                                                          Singular Values in Increasing Order

                                                                          Review of Linear Algebra (Cont)

                                                                          )min(1 dnss

                                                                          SVD Compact Representation

                                                                          Useful Labeling

                                                                          Singular Values in Increasing Order

                                                                          Note singular values = 0 can be omitted

                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                          Review of Linear Algebra (Cont)

                                                                          )min(1 dnss

                                                                          Review of Linear Algebra (Cont)

                                                                          r

                                                                          SVD Compact Representation

                                                                          Useful Labeling

                                                                          Singular Values in Increasing Order

                                                                          Note singular values = 0 can be omitted

                                                                          Let = of positive singular values

                                                                          )min(1 dnss

                                                                          Review of Linear Algebra (Cont)

                                                                          r

                                                                          SVD Compact Representation

                                                                          Useful Labeling

                                                                          Singular Values in Increasing Order

                                                                          Note singular values = 0 can be omitted

                                                                          Let = of positive singular values

                                                                          Then

                                                                          Where are truncations of

                                                                          trnrrrd VSUX

                                                                          VSU

                                                                          )min(1 dnss

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Full Representation

                                                                          =

                                                                          Graphics Display Assumes

                                                                          ndX ddU ndS nn

                                                                          tV

                                                                          nd

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Full Representation

                                                                          =

                                                                          Full Rank Basis Matrix

                                                                          ndX ddU ndS nn

                                                                          tV

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Full Representation

                                                                          =

                                                                          Full Rank Basis Matrix

                                                                          All 0s in Bottom

                                                                          ndX ddU ndS nn

                                                                          tV

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Reduced Representation

                                                                          =

                                                                          These Columns Get 0ed Out

                                                                          ndX ddU nnS nn

                                                                          tV

                                                                          nnd 0

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Reduced Representation

                                                                          =ndX ndU nnS nn

                                                                          tV

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Reduced Representation

                                                                          =

                                                                          Also Some of These May be 0

                                                                          ndX ndU nnS nn

                                                                          tV

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Compact Representation

                                                                          = ndX rdU

                                                                          rrS nrtV

                                                                          0

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Compact Representation

                                                                          =

                                                                          These Get 0ed Out

                                                                          ndX rdU

                                                                          rrS nrtV

                                                                          0

                                                                          Review of Linear Algebra (Cont)

                                                                          SVD Compact Representation

                                                                          = ndX rdU

                                                                          rrS nrtV

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition

                                                                          For a (Symmetric) Square Matrix

                                                                          Find

                                                                          ddX

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition

                                                                          For a (Symmetric) Square Matrix

                                                                          Find a Diagonal Matrix

                                                                          Called Eigenvalues

                                                                          Convenient Ordering

                                                                          ddX

                                                                          d

                                                                          D

                                                                          0

                                                                          01

                                                                          n 1

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition

                                                                          For a (Symmetric) Square Matrix

                                                                          Find a Diagonal Matrix

                                                                          And an Orthonormal Matrix

                                                                          (ie )

                                                                          ddX

                                                                          d

                                                                          D

                                                                          0

                                                                          01

                                                                          ddB

                                                                          ddtt IBBBB

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition

                                                                          For a (Symmetric) Square Matrix

                                                                          Find a Diagonal Matrix

                                                                          And an Orthonormal Matrix

                                                                          (ie )

                                                                          So that ie

                                                                          ddX

                                                                          d

                                                                          D

                                                                          0

                                                                          01

                                                                          ddB

                                                                          ddtt IBBBB

                                                                          DBBX tBDBX

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                          (looks similar)

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                          Complex (uses )

                                                                          VU

                                                                          1i

                                                                          Review of Linear Algebra (Cont)

                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                          VU

                                                                          X

                                                                          BVU

                                                                          1i

                                                                          • Participant Presentations
                                                                          • PCA to find clusters
                                                                          • Statistical Smoothing
                                                                          • Kernel Density Estimation
                                                                          • Scatterplot Smoothing
                                                                          • Statistical Smoothing (2)
                                                                          • SiZer Background
                                                                          • SiZer Background (2)
                                                                          • SiZer Background (3)
                                                                          • SiZer Background (4)
                                                                          • SiZer Background (5)
                                                                          • SiZer Overview
                                                                          • PCA to find clusters (2)
                                                                          • PCA to find clusters (3)
                                                                          • PCA to find clusters (4)
                                                                          • PCA to find clusters (5)
                                                                          • PCA to find clusters (6)
                                                                          • PCA to find clusters (7)
                                                                          • Recall Yeast Cell Cycle Data
                                                                          • Yeast Cell Cycle Data FDA View
                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                          • Frequency 2 Analysis
                                                                          • Frequency 2 Analysis (2)
                                                                          • Frequency 2 Analysis (3)
                                                                          • Yeast Cell Cycle
                                                                          • SiZer Study of Distrsquon of Angles
                                                                          • Reclassification of Major Genes
                                                                          • Compare to Previous Classifrsquon
                                                                          • New Subpopulation View
                                                                          • New Subpopulation View (2)
                                                                          • Detailed Look at PCA
                                                                          • PCA Rediscovery ndash Renaming
                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                          • An Interesting Historical Note
                                                                          • An Interesting Historical Note (2)
                                                                          • Detailed Look at PCA (2)
                                                                          • Detailed Look at PCA (3)
                                                                          • Review of Linear Algebra
                                                                          • Review of Linear Algebra (2)
                                                                          • Review of Linear Algebra (3)
                                                                          • Review of Linear Algebra (Cont)
                                                                          • Review of Linear Algebra (Cont) (2)
                                                                          • Review of Linear Algebra (Cont) (3)
                                                                          • Review of Linear Algebra (Cont) (4)
                                                                          • Review of Linear Algebra (Cont) (5)
                                                                          • Review of Linear Algebra (Cont) (6)
                                                                          • Review of Linear Algebra (Cont) (7)
                                                                          • Review of Linear Algebra (Cont) (8)
                                                                          • Review of Linear Algebra (Cont) (9)
                                                                          • Review of Linear Algebra (Cont) (10)
                                                                          • Review of Linear Algebra (Cont) (11)
                                                                          • Review of Linear Algebra (Cont) (12)
                                                                          • Review of Linear Algebra (Cont) (13)
                                                                          • Review of Linear Algebra (Cont) (14)
                                                                          • Review of Linear Algebra (Cont) (15)
                                                                          • Review of Linear Algebra (Cont) (16)
                                                                          • Review of Linear Algebra (Cont) (17)
                                                                          • Review of Linear Algebra (Cont) (18)
                                                                          • Review of Linear Algebra (Cont) (19)
                                                                          • Review of Linear Algebra (Cont) (20)
                                                                          • Review of Linear Algebra (Cont) (21)
                                                                          • Review of Linear Algebra (Cont) (22)
                                                                          • Review of Linear Algebra (Cont) (23)
                                                                          • Review of Linear Algebra (Cont) (24)
                                                                          • Review of Linear Algebra (Cont) (25)
                                                                          • Review of Linear Algebra (Cont) (26)
                                                                          • Review of Linear Algebra (Cont) (27)
                                                                          • Review of Linear Algebra (Cont) (28)
                                                                          • Review of Linear Algebra (Cont) (29)
                                                                          • Review of Linear Algebra (Cont) (30)
                                                                          • Review of Linear Algebra (Cont) (31)
                                                                          • Review of Linear Algebra (Cont) (32)
                                                                          • Review of Linear Algebra (Cont) (33)
                                                                          • Review of Linear Algebra (Cont) (34)
                                                                          • Review of Linear Algebra (Cont) (35)
                                                                          • Review of Linear Algebra (Cont) (36)
                                                                          • Review of Linear Algebra (Cont) (37)
                                                                          • Review of Linear Algebra (Cont) (38)
                                                                          • Review of Linear Algebra (Cont) (39)
                                                                          • Review of Linear Algebra (Cont) (40)
                                                                          • Review of Linear Algebra (Cont) (41)
                                                                          • Review of Linear Algebra (Cont) (42)
                                                                          • Review of Linear Algebra (Cont) (43)
                                                                          • Review of Linear Algebra (Cont) (44)
                                                                          • Review of Linear Algebra (Cont) (45)
                                                                          • Review of Linear Algebra (Cont) (46)
                                                                          • Review of Linear Algebra (Cont) (47)
                                                                          • Review of Linear Algebra (Cont) (48)
                                                                          • Review of Linear Algebra (Cont) (49)
                                                                          • Review of Linear Algebra (Cont) (50)
                                                                          • Review of Linear Algebra (Cont) (51)
                                                                          • Review of Linear Algebra (Cont) (52)
                                                                          • Review of Linear Algebra (Cont) (53)
                                                                          • Review of Linear Algebra (Cont) (54)
                                                                          • Review of Linear Algebra (Cont) (55)
                                                                          • Review of Linear Algebra (Cont) (56)
                                                                          • Review of Linear Algebra (Cont) (57)
                                                                          • Review of Linear Algebra (Cont) (58)
                                                                          • Review of Linear Algebra (Cont) (59)
                                                                          • Review of Linear Algebra (Cont) (60)
                                                                          • Review of Linear Algebra (Cont) (61)
                                                                          • Review of Linear Algebra (Cont) (62)
                                                                          • Review of Linear Algebra (Cont) (63)
                                                                          • Review of Linear Algebra (Cont) (64)
                                                                          • Review of Linear Algebra (Cont) (65)
                                                                          • Review of Linear Algebra (Cont) (66)
                                                                          • Review of Linear Algebra (Cont) (67)
                                                                          • Review of Linear Algebra (Cont) (68)
                                                                          • Review of Linear Algebra (Cont) (69)
                                                                          • Review of Linear Algebra (Cont) (70)
                                                                          • Review of Linear Algebra (Cont) (71)
                                                                          • Review of Linear Algebra (Cont) (72)
                                                                          • Review of Linear Algebra (Cont) (73)
                                                                          • Review of Linear Algebra (Cont) (74)
                                                                          • Review of Linear Algebra (Cont) (75)
                                                                          • Review of Linear Algebra (Cont) (76)
                                                                          • Review of Linear Algebra (Cont) (77)
                                                                          • Review of Linear Algebra (Cont) (78)
                                                                          • Review of Linear Algebra (Cont) (79)
                                                                          • Review of Linear Algebra (Cont) (80)
                                                                          • Review of Linear Algebra (Cont) (81)

                                                                            PCA Rediscovery ndash Renaming

                                                                            Statistics Principal Component Analysis (PCA)

                                                                            Social Sciences Factor Analysis (PCA is a subset)

                                                                            Probability Electrical EngKarhunen ndash Loeve expansion

                                                                            Applied MathematicsProper Orthogonal Decomposition (POD)

                                                                            Geo-Sciences Empirical Orthogonal Functions (EOF)

                                                                            An Interesting Historical Note

                                                                            The 1st () application of PCA to Functional

                                                                            Data Analysis

                                                                            An Interesting Historical Note

                                                                            The 1st () application of PCA to Functional

                                                                            Data Analysis

                                                                            Rao (1958)

                                                                            1st Paper with ldquoCurves as Data Objectsrdquo

                                                                            viewpoint

                                                                            Detailed Look at PCA

                                                                            Three Important (amp Interesting) Viewpoints

                                                                            1 Mathematics

                                                                            2 Numerics

                                                                            3 Statistics

                                                                            Goal Study Interrelationships

                                                                            Detailed Look at PCA

                                                                            Three Important (amp Interesting) Viewpoints

                                                                            1 Mathematics

                                                                            2 Numerics

                                                                            3 Statistics

                                                                            1st Review Linear Alg and Multivar Prob

                                                                            Review of Linear Algebra

                                                                            Vector Space

                                                                            bull set of ldquovectorsrdquo

                                                                            bull and ldquoscalarsrdquo (coefficients)

                                                                            xa

                                                                            Review of Linear Algebra

                                                                            Vector Space

                                                                            bull set of ldquovectorsrdquo

                                                                            bull and ldquoscalarsrdquo (coefficients)

                                                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                            ( in space)

                                                                            xa

                                                                            i

                                                                            ii xa

                                                                            Review of Linear Algebra

                                                                            Vector Space

                                                                            bull set of ldquovectorsrdquo

                                                                            bull and ldquoscalarsrdquo (coefficients)

                                                                            bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                            ( in space)

                                                                            eg

                                                                            ldquo dim Euclidrsquon spacerdquo

                                                                            xa

                                                                            i

                                                                            ii xa

                                                                            d

                                                                            d

                                                                            d xx

                                                                            x

                                                                            x

                                                                            x 1

                                                                            1

                                                                            d

                                                                            Review of Linear Algebra (Cont)

                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                            Review of Linear Algebra (Cont)

                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                            Review of Linear Algebra (Cont)

                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                            Note Planes not Through the Origin

                                                                            are not Subspaces

                                                                            (Do not Contain )00 x

                                                                            Review of Linear Algebra (Cont)

                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                            (all Linear Combos of them =

                                                                            = Containing Hyperplane

                                                                            through Origin)

                                                                            Review of Linear Algebra (Cont)

                                                                            Basis of Subspace Set of Vectors that

                                                                            bull Span ie Everything is a Lin Com of them

                                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                            Review of Linear Algebra (Cont)

                                                                            Basis of Subspace Set of Vectors that

                                                                            bull Span ie Everything is a Lin Com of them

                                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                            bull eg ldquoUnit Vector Basisrdquo

                                                                            bull Since

                                                                            d

                                                                            1

                                                                            0

                                                                            0

                                                                            0

                                                                            1

                                                                            0

                                                                            0

                                                                            0

                                                                            1

                                                                            1

                                                                            0

                                                                            0

                                                                            0

                                                                            1

                                                                            0

                                                                            0

                                                                            0

                                                                            1

                                                                            212

                                                                            1

                                                                            d

                                                                            d

                                                                            xxx

                                                                            x

                                                                            x

                                                                            x

                                                                            Review of Linear Algebra (Cont)

                                                                            Basis Matrix of subspace of

                                                                            Given a basis

                                                                            create matrix of columns

                                                                            dnvv 1

                                                                            nddnd

                                                                            n

                                                                            n

                                                                            vv

                                                                            vv

                                                                            vvB

                                                                            1

                                                                            111

                                                                            1

                                                                            Review of Linear Algebra (Cont)

                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                            where

                                                                            n

                                                                            iii aBva

                                                                            1

                                                                            na

                                                                            a

                                                                            a 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                            where

                                                                            Note Right Multiplication Gives

                                                                            Linear Combination of Column Vectors

                                                                            n

                                                                            iii aBva

                                                                            1

                                                                            na

                                                                            a

                                                                            a 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                            where

                                                                            Check sizes

                                                                            n

                                                                            iii aBva

                                                                            1

                                                                            na

                                                                            a

                                                                            a 1

                                                                            )1()(1 nndd

                                                                            Review of Linear Algebra (Cont)

                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                            For matrices

                                                                            Define the Matrix Product

                                                                            mkk

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            nmm

                                                                            n

                                                                            bb

                                                                            bb

                                                                            B

                                                                            1

                                                                            111

                                                                            m

                                                                            iniik

                                                                            m

                                                                            iiik

                                                                            m

                                                                            inii

                                                                            m

                                                                            iii

                                                                            baba

                                                                            baba

                                                                            AB

                                                                            1

                                                                            11

                                                                            11

                                                                            111

                                                                            Review of Linear Algebra (Cont)

                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                            For matrices

                                                                            Define the Matrix Product

                                                                            (Inner Products of Rows With Columns )

                                                                            mkk

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            nmm

                                                                            n

                                                                            bb

                                                                            bb

                                                                            B

                                                                            1

                                                                            111

                                                                            m

                                                                            iniik

                                                                            m

                                                                            iiik

                                                                            m

                                                                            inii

                                                                            m

                                                                            iii

                                                                            baba

                                                                            baba

                                                                            AB

                                                                            1

                                                                            11

                                                                            11

                                                                            111

                                                                            A B

                                                                            Review of Linear Algebra (Cont)

                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                            For matrices

                                                                            Define the Matrix Product

                                                                            (Inner Products of Rows With Columns )

                                                                            (Composition of Linear Transformations)

                                                                            mkk

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            nmm

                                                                            n

                                                                            bb

                                                                            bb

                                                                            B

                                                                            1

                                                                            111

                                                                            m

                                                                            iniik

                                                                            m

                                                                            iiik

                                                                            m

                                                                            inii

                                                                            m

                                                                            iii

                                                                            baba

                                                                            baba

                                                                            AB

                                                                            1

                                                                            11

                                                                            11

                                                                            111

                                                                            A B

                                                                            Review of Linear Algebra (Cont)

                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                            For matrices

                                                                            Define the Matrix Product

                                                                            (Inner Products of Rows With Columns )

                                                                            (Composition of Linear Transformations)

                                                                            Often Useful to Check Sizes

                                                                            mkk

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            nmm

                                                                            n

                                                                            bb

                                                                            bb

                                                                            B

                                                                            1

                                                                            111

                                                                            m

                                                                            iniik

                                                                            m

                                                                            iiik

                                                                            m

                                                                            inii

                                                                            m

                                                                            iii

                                                                            baba

                                                                            baba

                                                                            AB

                                                                            1

                                                                            11

                                                                            11

                                                                            111

                                                                            nmmknk

                                                                            A B

                                                                            Review of Linear Algebra (Cont)

                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                            For matrices

                                                                            Define the Matrix Product

                                                                            (Inner Products of Rows With Columns )

                                                                            (Composition of Linear Transformations)

                                                                            Often Useful to Check Sizes

                                                                            mkk

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            nmm

                                                                            n

                                                                            bb

                                                                            bb

                                                                            B

                                                                            1

                                                                            111

                                                                            m

                                                                            iniik

                                                                            m

                                                                            iiik

                                                                            m

                                                                            inii

                                                                            m

                                                                            iii

                                                                            baba

                                                                            baba

                                                                            AB

                                                                            1

                                                                            11

                                                                            11

                                                                            111

                                                                            nmmknk

                                                                            A B

                                                                            Review of Linear Algebra (Cont)

                                                                            Matrix Trace

                                                                            bull For a Square Matrix

                                                                            bull Define

                                                                            mmm

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            m

                                                                            iiiaAtr

                                                                            1)(

                                                                            Review of Linear Algebra (Cont)

                                                                            Matrix Trace

                                                                            bull For a Square Matrix

                                                                            bull Define

                                                                            bull Trace Commutes with Matrix Multiplication

                                                                            mmm

                                                                            m

                                                                            aa

                                                                            aa

                                                                            A

                                                                            1

                                                                            111

                                                                            m

                                                                            iiiaAtr

                                                                            1)(

                                                                            BAtrABtr

                                                                            Review of Linear Algebra (Cont)

                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                            bull Number of Elements in a Basis (Unique)

                                                                            Review of Linear Algebra (Cont)

                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                            bull Number of Elements in a Basis (Unique)

                                                                            bull (Use Basis Above)

                                                                            bull eg dim of a line is 1

                                                                            bull eg dim of a plane is 2

                                                                            dd dim

                                                                            Review of Linear Algebra (Cont)

                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                            bull Number of Elements in a Basis (Unique)

                                                                            bull (Use Basis Above)

                                                                            bull eg dim of a line is 1

                                                                            bull eg dim of a plane is 2

                                                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                                                            (in Statistical Uses eg ANOVA)

                                                                            dd dim

                                                                            Review of Linear Algebra (Cont)

                                                                            Norm of a Vector

                                                                            bull in d 21

                                                                            21

                                                                            1

                                                                            2 xxxx td

                                                                            jj

                                                                            Review of Linear Algebra (Cont)

                                                                            Norm of a Vector

                                                                            bull in

                                                                            bull Idea length of the vector

                                                                            d 2121

                                                                            1

                                                                            2 xxxx td

                                                                            jj

                                                                            Review of Linear Algebra (Cont)

                                                                            Norm of a Vector

                                                                            bull in

                                                                            bull Idea length of the vector

                                                                            bull Note strange properties for high

                                                                            eg ldquolength of diagonal of unit cuberdquo =

                                                                            d 2121

                                                                            1

                                                                            2 xxxx td

                                                                            jj

                                                                            d

                                                                            d

                                                                            Review of Linear Algebra (Cont)

                                                                            Norm of a Vector (cont)

                                                                            bull Length Normalized Vector

                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                            amp is a Direction Vector)

                                                                            x

                                                                            x

                                                                            Review of Linear Algebra (Cont)

                                                                            Norm of a Vector (cont)

                                                                            bull Length Normalized Vector

                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                            amp is a Direction Vector)

                                                                            bull Define Distance as

                                                                            x

                                                                            x

                                                                            yxyxyxyxd t

                                                                            Review of Linear Algebra (Cont)

                                                                            Inner (Dot Scalar) Product

                                                                            bull for Vectors and

                                                                            yxyxyx td

                                                                            jjj

                                                                            1

                                                                            x y

                                                                            Review of Linear Algebra (Cont)

                                                                            Inner (Dot Scalar) Product

                                                                            bull for Vectors and

                                                                            bull Related to Norm via

                                                                            yxyxyx td

                                                                            jjj

                                                                            1

                                                                            x y

                                                                            21 xxx

                                                                            Review of Linear Algebra (Cont)

                                                                            Inner (Dot Scalar) Product (cont)

                                                                            bull measures ldquoangle between and rdquo as

                                                                            yyxx

                                                                            yx

                                                                            yx

                                                                            yxyxangle

                                                                            tt

                                                                            t

                                                                            11 cos

                                                                            cos

                                                                            x y

                                                                            Review of Linear Algebra (Cont)

                                                                            Inner (Dot Scalar) Product (cont)

                                                                            bull measures ldquoangle between and rdquo as

                                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                                            if and only if

                                                                            yyxx

                                                                            yx

                                                                            yx

                                                                            yxyxangle

                                                                            tt

                                                                            t

                                                                            11 cos

                                                                            cos

                                                                            x y

                                                                            yx 0 yx

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis

                                                                            bull All Orthogonal to each other

                                                                            ie for

                                                                            bull All have Length 1

                                                                            ie for

                                                                            nvv 1

                                                                            1 ii vv

                                                                            0 ii vv ii

                                                                            ni 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis (cont)

                                                                            bull Spectral Representation

                                                                            where

                                                                            nvv 1

                                                                            n

                                                                            iii vax

                                                                            1

                                                                            ii vxa

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis (cont)

                                                                            bull Spectral Representation

                                                                            where

                                                                            (Coefficient is Inner Product Cool Notation)

                                                                            nvv 1

                                                                            n

                                                                            iii vax

                                                                            1

                                                                            ii vxa

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis (cont)

                                                                            bull Spectral Representation

                                                                            where

                                                                            Check

                                                                            nvv 1

                                                                            n

                                                                            iii vax

                                                                            1

                                                                            ii vxa

                                                                            iii

                                                                            n

                                                                            iii

                                                                            n

                                                                            iiii avvavvavx

                                                                            1

                                                                            1

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis (cont)

                                                                            bull Spectral Representation

                                                                            where

                                                                            Check

                                                                            bull Matrix Notation where ie

                                                                            For the Basis Matrix

                                                                            nvv 1

                                                                            n

                                                                            iii vax

                                                                            1

                                                                            ii vxa

                                                                            iii

                                                                            n

                                                                            iii

                                                                            n

                                                                            iiii avvavvavx

                                                                            1

                                                                            1

                                                                            aBx Bxa tt xBa t

                                                                            nvvB 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Orthonormal Basis (cont)

                                                                            bull Spectral Representation

                                                                            where

                                                                            Check

                                                                            bull Matrix Notation where ie

                                                                            is called transform of

                                                                            (eg Fourier or Wavelet)

                                                                            nvv 1

                                                                            n

                                                                            iii vax

                                                                            1

                                                                            ii vxa

                                                                            iii

                                                                            n

                                                                            iii

                                                                            n

                                                                            iiii avvavvavx

                                                                            1

                                                                            1

                                                                            aBx Bxa tt xBa t

                                                                            xa

                                                                            Review of Linear Algebra (Cont)

                                                                            Parseval identity for

                                                                            in subsp genrsquod by o n basis

                                                                            x

                                                                            nvv 1

                                                                            2

                                                                            1

                                                                            22

                                                                            1

                                                                            2 aavxx

                                                                            n

                                                                            ii

                                                                            n

                                                                            ii

                                                                            Review of Linear Algebra (Cont)

                                                                            Parseval identity for

                                                                            in subsp genrsquod by o n basis

                                                                            bull Pythagorean theorem

                                                                            bull ldquoDecomposition of Energyrdquo

                                                                            bull ANOVA - sums of squares

                                                                            x

                                                                            nvv 1

                                                                            2

                                                                            1

                                                                            22

                                                                            1

                                                                            2 aavxx

                                                                            n

                                                                            ii

                                                                            n

                                                                            ii

                                                                            Review of Linear Algebra (Cont)

                                                                            Parseval identity for

                                                                            in subsp genrsquod by o n basis

                                                                            bull Pythagorean theorem

                                                                            bull ldquoDecomposition of Energyrdquo

                                                                            bull ANOVA - sums of squares

                                                                            bull Transform has same length as

                                                                            ie ldquorotation in rdquo

                                                                            x

                                                                            nvv 1

                                                                            2

                                                                            1

                                                                            22

                                                                            1

                                                                            2 aavxx

                                                                            n

                                                                            ii

                                                                            n

                                                                            ii

                                                                            a xd

                                                                            Projection of a Vector onto a Subspace

                                                                            bull Idea Member of that is Closest to

                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                            Review of Linear Algebra (Cont)x

                                                                            xV

                                                                            V

                                                                            Projection of a Vector onto a Subspace

                                                                            bull Idea Member of that is Closest to

                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                            bull Find that Solves

                                                                            (ldquoLeast Squaresrdquo)

                                                                            Review of Linear Algebra (Cont)x

                                                                            xV

                                                                            V

                                                                            VxPV vxVv

                                                                            min

                                                                            Projection of a Vector onto a Subspace

                                                                            bull Idea Member of that is Closest to

                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                            bull Find that Solves

                                                                            (ldquoLeast Squaresrdquo)

                                                                            bull For Inner Product (Hilbert) Space

                                                                            Exists and is Unique

                                                                            Review of Linear Algebra (Cont)x

                                                                            xV

                                                                            V

                                                                            VxPV vxVv

                                                                            min

                                                                            xPV

                                                                            Projection of a Vector onto a Subspace (cont)

                                                                            bull General Solution in for Basis Matrix

                                                                            Review of Linear Algebra (Cont)

                                                                            d VB

                                                                            xBBBBxP tVV

                                                                            tVVV

                                                                            1

                                                                            Projection of a Vector onto a Subspace (cont)

                                                                            bull General Solution in for Basis Matrix

                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                            (thus projection is another linear operation)

                                                                            Review of Linear Algebra (Cont)

                                                                            d VB

                                                                            xBBBBxP tVV

                                                                            tVVV

                                                                            1

                                                                            tVV

                                                                            tVVV BBBBP

                                                                            1

                                                                            Projection of a Vector onto a Subspace (cont)

                                                                            bull General Solution in for Basis Matrix

                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                            (thus projection is another linear operation)

                                                                            (note same operation underlies least squares)

                                                                            Review of Linear Algebra (Cont)

                                                                            d VB

                                                                            xBBBBxP tVV

                                                                            tVVV

                                                                            1

                                                                            tVV

                                                                            tVVV BBBBP

                                                                            1

                                                                            Review of Linear Algebra (Cont)

                                                                            Projection using Orthonormal Basis

                                                                            bull Basis Matrix is Orthonormal

                                                                            nnVtV IBB

                                                                            10

                                                                            01

                                                                            1

                                                                            111

                                                                            1

                                                                            1

                                                                            nnn

                                                                            n

                                                                            ntn

                                                                            t

                                                                            vvvv

                                                                            vvvv

                                                                            vv

                                                                            v

                                                                            v

                                                                            nvv 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Projection using Orthonormal Basis

                                                                            bull Basis Matrix is Orthonormal

                                                                            bull So =

                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                            (Recall Right Multrsquon)

                                                                            nnVtV IBB

                                                                            10

                                                                            01

                                                                            1

                                                                            111

                                                                            1

                                                                            1

                                                                            nnn

                                                                            n

                                                                            ntn

                                                                            t

                                                                            vvvv

                                                                            vvvv

                                                                            vv

                                                                            v

                                                                            v

                                                                            xBBxP tVVV

                                                                            x V

                                                                            nvv 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Projection using Orthonormal Basis (cont)

                                                                            bull For Orthogonal Complement

                                                                            and

                                                                            V

                                                                            xPxPx VV 222xPxPx VV

                                                                            Review of Linear Algebra (Cont)

                                                                            Projection using Orthonormal Basis (cont)

                                                                            bull For Orthogonal Complement

                                                                            and

                                                                            bull Parseval Inequality

                                                                            V

                                                                            xPxPx VV 222xPxPx VV

                                                                            2

                                                                            1

                                                                            22

                                                                            1

                                                                            22 aavxxxP

                                                                            n

                                                                            ii

                                                                            n

                                                                            iiV

                                                                            Review of Linear Algebra (Cont)

                                                                            (Real) Unitary Matrices with

                                                                            bull Orthonormal Basis Matrix

                                                                            (So All of Above Applies)

                                                                            ddU IUU t

                                                                            Review of Linear Algebra (Cont)

                                                                            (Real) Unitary Matrices with

                                                                            bull Orthonormal Basis Matrix

                                                                            (So All of Above Applies)

                                                                            bull Note Transformrsquon is Distance Preserving

                                                                            ddU IUU t

                                                                            yxdyxyxyxUyUxUdn

                                                                            i ii 2

                                                                            1

                                                                            Review of Linear Algebra (Cont)

                                                                            (Real) Unitary Matrices with

                                                                            bull Orthonormal Basis Matrix

                                                                            (So All of Above Applies)

                                                                            bull Note Transformrsquon is Distance Preserving

                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                            ddU IUU t

                                                                            yxdyxyxyxUyUxUdn

                                                                            i ii 2

                                                                            1

                                                                            U

                                                                            Review of Linear Algebra (Cont)

                                                                            Singular Value Decomposition (SVD)

                                                                            For a Matrix

                                                                            Find

                                                                            ndX

                                                                            Review of Linear Algebra (Cont)

                                                                            Singular Value Decomposition (SVD)

                                                                            For a Matrix

                                                                            Find a Diagonal Matrix

                                                                            with Entries

                                                                            called Singular Values

                                                                            ndX

                                                                            ndS

                                                                            )min(1 ndss

                                                                            Review of Linear Algebra (Cont)

                                                                            Singular Value Decomposition (SVD)

                                                                            For a Matrix

                                                                            Find a Diagonal Matrix

                                                                            with Entries

                                                                            called Singular Values

                                                                            And Unitary (Rotation) Matrices

                                                                            (recall )

                                                                            ndX

                                                                            ndS

                                                                            )min(1 ndss

                                                                            ddU nnV

                                                                            IVVUU tt

                                                                            Review of Linear Algebra (Cont)

                                                                            Singular Value Decomposition (SVD)

                                                                            For a Matrix

                                                                            Find a Diagonal Matrix

                                                                            with Entries

                                                                            called Singular Values

                                                                            And Unitary (Rotation) Matrices

                                                                            (recall )

                                                                            So That

                                                                            ndX

                                                                            ndS

                                                                            )min(1 ndss

                                                                            ddU nnV

                                                                            IVVUU tt tUSVX

                                                                            Review of Linear Algebra (Cont)

                                                                            Intuition behind Singular Value Decomposition

                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                            vVSUvVSUvX tt

                                                                            Review of Linear Algebra (Cont)

                                                                            Intuition behind Singular Value Decomposition

                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                            bull First rotate

                                                                            X

                                                                            vVSUvVSUvX tt

                                                                            Review of Linear Algebra (Cont)

                                                                            Intuition behind Singular Value Decomposition

                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                            bull First rotate

                                                                            bull Second rescale coordinate axes (by )

                                                                            X

                                                                            vVSUvVSUvX tt

                                                                            is

                                                                            Review of Linear Algebra (Cont)

                                                                            Intuition behind Singular Value Decomposition

                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                            bull First rotate

                                                                            bull Second rescale coordinate axes (by )

                                                                            bull Third rotate again

                                                                            X

                                                                            vVSUvVSUvX tt

                                                                            is

                                                                            Review of Linear Algebra (Cont)

                                                                            Intuition behind Singular Value Decomposition

                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                            bull First rotate

                                                                            bull Second rescale coordinate axes (by )

                                                                            bull Third rotate again

                                                                            bull ie have diagonalized the transformation

                                                                            X

                                                                            vVSUvVSUvX tt

                                                                            is

                                                                            SVD Compact Representation

                                                                            Useful Labeling

                                                                            Singular Values in Increasing Order

                                                                            Review of Linear Algebra (Cont)

                                                                            )min(1 dnss

                                                                            SVD Compact Representation

                                                                            Useful Labeling

                                                                            Singular Values in Increasing Order

                                                                            Note singular values = 0 can be omitted

                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                            Review of Linear Algebra (Cont)

                                                                            )min(1 dnss

                                                                            Review of Linear Algebra (Cont)

                                                                            r

                                                                            SVD Compact Representation

                                                                            Useful Labeling

                                                                            Singular Values in Increasing Order

                                                                            Note singular values = 0 can be omitted

                                                                            Let = of positive singular values

                                                                            )min(1 dnss

                                                                            Review of Linear Algebra (Cont)

                                                                            r

                                                                            SVD Compact Representation

                                                                            Useful Labeling

                                                                            Singular Values in Increasing Order

                                                                            Note singular values = 0 can be omitted

                                                                            Let = of positive singular values

                                                                            Then

                                                                            Where are truncations of

                                                                            trnrrrd VSUX

                                                                            VSU

                                                                            )min(1 dnss

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Full Representation

                                                                            =

                                                                            Graphics Display Assumes

                                                                            ndX ddU ndS nn

                                                                            tV

                                                                            nd

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Full Representation

                                                                            =

                                                                            Full Rank Basis Matrix

                                                                            ndX ddU ndS nn

                                                                            tV

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Full Representation

                                                                            =

                                                                            Full Rank Basis Matrix

                                                                            All 0s in Bottom

                                                                            ndX ddU ndS nn

                                                                            tV

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Reduced Representation

                                                                            =

                                                                            These Columns Get 0ed Out

                                                                            ndX ddU nnS nn

                                                                            tV

                                                                            nnd 0

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Reduced Representation

                                                                            =ndX ndU nnS nn

                                                                            tV

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Reduced Representation

                                                                            =

                                                                            Also Some of These May be 0

                                                                            ndX ndU nnS nn

                                                                            tV

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Compact Representation

                                                                            = ndX rdU

                                                                            rrS nrtV

                                                                            0

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Compact Representation

                                                                            =

                                                                            These Get 0ed Out

                                                                            ndX rdU

                                                                            rrS nrtV

                                                                            0

                                                                            Review of Linear Algebra (Cont)

                                                                            SVD Compact Representation

                                                                            = ndX rdU

                                                                            rrS nrtV

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition

                                                                            For a (Symmetric) Square Matrix

                                                                            Find

                                                                            ddX

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition

                                                                            For a (Symmetric) Square Matrix

                                                                            Find a Diagonal Matrix

                                                                            Called Eigenvalues

                                                                            Convenient Ordering

                                                                            ddX

                                                                            d

                                                                            D

                                                                            0

                                                                            01

                                                                            n 1

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition

                                                                            For a (Symmetric) Square Matrix

                                                                            Find a Diagonal Matrix

                                                                            And an Orthonormal Matrix

                                                                            (ie )

                                                                            ddX

                                                                            d

                                                                            D

                                                                            0

                                                                            01

                                                                            ddB

                                                                            ddtt IBBBB

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition

                                                                            For a (Symmetric) Square Matrix

                                                                            Find a Diagonal Matrix

                                                                            And an Orthonormal Matrix

                                                                            (ie )

                                                                            So that ie

                                                                            ddX

                                                                            d

                                                                            D

                                                                            0

                                                                            01

                                                                            ddB

                                                                            ddtt IBBBB

                                                                            DBBX tBDBX

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                            (looks similar)

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                            Complex (uses )

                                                                            VU

                                                                            1i

                                                                            Review of Linear Algebra (Cont)

                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                            VU

                                                                            X

                                                                            BVU

                                                                            1i

                                                                            • Participant Presentations
                                                                            • PCA to find clusters
                                                                            • Statistical Smoothing
                                                                            • Kernel Density Estimation
                                                                            • Scatterplot Smoothing
                                                                            • Statistical Smoothing (2)
                                                                            • SiZer Background
                                                                            • SiZer Background (2)
                                                                            • SiZer Background (3)
                                                                            • SiZer Background (4)
                                                                            • SiZer Background (5)
                                                                            • SiZer Overview
                                                                            • PCA to find clusters (2)
                                                                            • PCA to find clusters (3)
                                                                            • PCA to find clusters (4)
                                                                            • PCA to find clusters (5)
                                                                            • PCA to find clusters (6)
                                                                            • PCA to find clusters (7)
                                                                            • Recall Yeast Cell Cycle Data
                                                                            • Yeast Cell Cycle Data FDA View
                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                            • Frequency 2 Analysis
                                                                            • Frequency 2 Analysis (2)
                                                                            • Frequency 2 Analysis (3)
                                                                            • Yeast Cell Cycle
                                                                            • SiZer Study of Distrsquon of Angles
                                                                            • Reclassification of Major Genes
                                                                            • Compare to Previous Classifrsquon
                                                                            • New Subpopulation View
                                                                            • New Subpopulation View (2)
                                                                            • Detailed Look at PCA
                                                                            • PCA Rediscovery ndash Renaming
                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                            • An Interesting Historical Note
                                                                            • An Interesting Historical Note (2)
                                                                            • Detailed Look at PCA (2)
                                                                            • Detailed Look at PCA (3)
                                                                            • Review of Linear Algebra
                                                                            • Review of Linear Algebra (2)
                                                                            • Review of Linear Algebra (3)
                                                                            • Review of Linear Algebra (Cont)
                                                                            • Review of Linear Algebra (Cont) (2)
                                                                            • Review of Linear Algebra (Cont) (3)
                                                                            • Review of Linear Algebra (Cont) (4)
                                                                            • Review of Linear Algebra (Cont) (5)
                                                                            • Review of Linear Algebra (Cont) (6)
                                                                            • Review of Linear Algebra (Cont) (7)
                                                                            • Review of Linear Algebra (Cont) (8)
                                                                            • Review of Linear Algebra (Cont) (9)
                                                                            • Review of Linear Algebra (Cont) (10)
                                                                            • Review of Linear Algebra (Cont) (11)
                                                                            • Review of Linear Algebra (Cont) (12)
                                                                            • Review of Linear Algebra (Cont) (13)
                                                                            • Review of Linear Algebra (Cont) (14)
                                                                            • Review of Linear Algebra (Cont) (15)
                                                                            • Review of Linear Algebra (Cont) (16)
                                                                            • Review of Linear Algebra (Cont) (17)
                                                                            • Review of Linear Algebra (Cont) (18)
                                                                            • Review of Linear Algebra (Cont) (19)
                                                                            • Review of Linear Algebra (Cont) (20)
                                                                            • Review of Linear Algebra (Cont) (21)
                                                                            • Review of Linear Algebra (Cont) (22)
                                                                            • Review of Linear Algebra (Cont) (23)
                                                                            • Review of Linear Algebra (Cont) (24)
                                                                            • Review of Linear Algebra (Cont) (25)
                                                                            • Review of Linear Algebra (Cont) (26)
                                                                            • Review of Linear Algebra (Cont) (27)
                                                                            • Review of Linear Algebra (Cont) (28)
                                                                            • Review of Linear Algebra (Cont) (29)
                                                                            • Review of Linear Algebra (Cont) (30)
                                                                            • Review of Linear Algebra (Cont) (31)
                                                                            • Review of Linear Algebra (Cont) (32)
                                                                            • Review of Linear Algebra (Cont) (33)
                                                                            • Review of Linear Algebra (Cont) (34)
                                                                            • Review of Linear Algebra (Cont) (35)
                                                                            • Review of Linear Algebra (Cont) (36)
                                                                            • Review of Linear Algebra (Cont) (37)
                                                                            • Review of Linear Algebra (Cont) (38)
                                                                            • Review of Linear Algebra (Cont) (39)
                                                                            • Review of Linear Algebra (Cont) (40)
                                                                            • Review of Linear Algebra (Cont) (41)
                                                                            • Review of Linear Algebra (Cont) (42)
                                                                            • Review of Linear Algebra (Cont) (43)
                                                                            • Review of Linear Algebra (Cont) (44)
                                                                            • Review of Linear Algebra (Cont) (45)
                                                                            • Review of Linear Algebra (Cont) (46)
                                                                            • Review of Linear Algebra (Cont) (47)
                                                                            • Review of Linear Algebra (Cont) (48)
                                                                            • Review of Linear Algebra (Cont) (49)
                                                                            • Review of Linear Algebra (Cont) (50)
                                                                            • Review of Linear Algebra (Cont) (51)
                                                                            • Review of Linear Algebra (Cont) (52)
                                                                            • Review of Linear Algebra (Cont) (53)
                                                                            • Review of Linear Algebra (Cont) (54)
                                                                            • Review of Linear Algebra (Cont) (55)
                                                                            • Review of Linear Algebra (Cont) (56)
                                                                            • Review of Linear Algebra (Cont) (57)
                                                                            • Review of Linear Algebra (Cont) (58)
                                                                            • Review of Linear Algebra (Cont) (59)
                                                                            • Review of Linear Algebra (Cont) (60)
                                                                            • Review of Linear Algebra (Cont) (61)
                                                                            • Review of Linear Algebra (Cont) (62)
                                                                            • Review of Linear Algebra (Cont) (63)
                                                                            • Review of Linear Algebra (Cont) (64)
                                                                            • Review of Linear Algebra (Cont) (65)
                                                                            • Review of Linear Algebra (Cont) (66)
                                                                            • Review of Linear Algebra (Cont) (67)
                                                                            • Review of Linear Algebra (Cont) (68)
                                                                            • Review of Linear Algebra (Cont) (69)
                                                                            • Review of Linear Algebra (Cont) (70)
                                                                            • Review of Linear Algebra (Cont) (71)
                                                                            • Review of Linear Algebra (Cont) (72)
                                                                            • Review of Linear Algebra (Cont) (73)
                                                                            • Review of Linear Algebra (Cont) (74)
                                                                            • Review of Linear Algebra (Cont) (75)
                                                                            • Review of Linear Algebra (Cont) (76)
                                                                            • Review of Linear Algebra (Cont) (77)
                                                                            • Review of Linear Algebra (Cont) (78)
                                                                            • Review of Linear Algebra (Cont) (79)
                                                                            • Review of Linear Algebra (Cont) (80)
                                                                            • Review of Linear Algebra (Cont) (81)

                                                                              An Interesting Historical Note

                                                                              The 1st () application of PCA to Functional

                                                                              Data Analysis

                                                                              An Interesting Historical Note

                                                                              The 1st () application of PCA to Functional

                                                                              Data Analysis

                                                                              Rao (1958)

                                                                              1st Paper with ldquoCurves as Data Objectsrdquo

                                                                              viewpoint

                                                                              Detailed Look at PCA

                                                                              Three Important (amp Interesting) Viewpoints

                                                                              1 Mathematics

                                                                              2 Numerics

                                                                              3 Statistics

                                                                              Goal Study Interrelationships

                                                                              Detailed Look at PCA

                                                                              Three Important (amp Interesting) Viewpoints

                                                                              1 Mathematics

                                                                              2 Numerics

                                                                              3 Statistics

                                                                              1st Review Linear Alg and Multivar Prob

                                                                              Review of Linear Algebra

                                                                              Vector Space

                                                                              bull set of ldquovectorsrdquo

                                                                              bull and ldquoscalarsrdquo (coefficients)

                                                                              xa

                                                                              Review of Linear Algebra

                                                                              Vector Space

                                                                              bull set of ldquovectorsrdquo

                                                                              bull and ldquoscalarsrdquo (coefficients)

                                                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                              ( in space)

                                                                              xa

                                                                              i

                                                                              ii xa

                                                                              Review of Linear Algebra

                                                                              Vector Space

                                                                              bull set of ldquovectorsrdquo

                                                                              bull and ldquoscalarsrdquo (coefficients)

                                                                              bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                              ( in space)

                                                                              eg

                                                                              ldquo dim Euclidrsquon spacerdquo

                                                                              xa

                                                                              i

                                                                              ii xa

                                                                              d

                                                                              d

                                                                              d xx

                                                                              x

                                                                              x

                                                                              x 1

                                                                              1

                                                                              d

                                                                              Review of Linear Algebra (Cont)

                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                              Review of Linear Algebra (Cont)

                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                              Review of Linear Algebra (Cont)

                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                              Note Planes not Through the Origin

                                                                              are not Subspaces

                                                                              (Do not Contain )00 x

                                                                              Review of Linear Algebra (Cont)

                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                              (all Linear Combos of them =

                                                                              = Containing Hyperplane

                                                                              through Origin)

                                                                              Review of Linear Algebra (Cont)

                                                                              Basis of Subspace Set of Vectors that

                                                                              bull Span ie Everything is a Lin Com of them

                                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                              Review of Linear Algebra (Cont)

                                                                              Basis of Subspace Set of Vectors that

                                                                              bull Span ie Everything is a Lin Com of them

                                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                              bull eg ldquoUnit Vector Basisrdquo

                                                                              bull Since

                                                                              d

                                                                              1

                                                                              0

                                                                              0

                                                                              0

                                                                              1

                                                                              0

                                                                              0

                                                                              0

                                                                              1

                                                                              1

                                                                              0

                                                                              0

                                                                              0

                                                                              1

                                                                              0

                                                                              0

                                                                              0

                                                                              1

                                                                              212

                                                                              1

                                                                              d

                                                                              d

                                                                              xxx

                                                                              x

                                                                              x

                                                                              x

                                                                              Review of Linear Algebra (Cont)

                                                                              Basis Matrix of subspace of

                                                                              Given a basis

                                                                              create matrix of columns

                                                                              dnvv 1

                                                                              nddnd

                                                                              n

                                                                              n

                                                                              vv

                                                                              vv

                                                                              vvB

                                                                              1

                                                                              111

                                                                              1

                                                                              Review of Linear Algebra (Cont)

                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                              where

                                                                              n

                                                                              iii aBva

                                                                              1

                                                                              na

                                                                              a

                                                                              a 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                              where

                                                                              Note Right Multiplication Gives

                                                                              Linear Combination of Column Vectors

                                                                              n

                                                                              iii aBva

                                                                              1

                                                                              na

                                                                              a

                                                                              a 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                              where

                                                                              Check sizes

                                                                              n

                                                                              iii aBva

                                                                              1

                                                                              na

                                                                              a

                                                                              a 1

                                                                              )1()(1 nndd

                                                                              Review of Linear Algebra (Cont)

                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                              For matrices

                                                                              Define the Matrix Product

                                                                              mkk

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              nmm

                                                                              n

                                                                              bb

                                                                              bb

                                                                              B

                                                                              1

                                                                              111

                                                                              m

                                                                              iniik

                                                                              m

                                                                              iiik

                                                                              m

                                                                              inii

                                                                              m

                                                                              iii

                                                                              baba

                                                                              baba

                                                                              AB

                                                                              1

                                                                              11

                                                                              11

                                                                              111

                                                                              Review of Linear Algebra (Cont)

                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                              For matrices

                                                                              Define the Matrix Product

                                                                              (Inner Products of Rows With Columns )

                                                                              mkk

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              nmm

                                                                              n

                                                                              bb

                                                                              bb

                                                                              B

                                                                              1

                                                                              111

                                                                              m

                                                                              iniik

                                                                              m

                                                                              iiik

                                                                              m

                                                                              inii

                                                                              m

                                                                              iii

                                                                              baba

                                                                              baba

                                                                              AB

                                                                              1

                                                                              11

                                                                              11

                                                                              111

                                                                              A B

                                                                              Review of Linear Algebra (Cont)

                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                              For matrices

                                                                              Define the Matrix Product

                                                                              (Inner Products of Rows With Columns )

                                                                              (Composition of Linear Transformations)

                                                                              mkk

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              nmm

                                                                              n

                                                                              bb

                                                                              bb

                                                                              B

                                                                              1

                                                                              111

                                                                              m

                                                                              iniik

                                                                              m

                                                                              iiik

                                                                              m

                                                                              inii

                                                                              m

                                                                              iii

                                                                              baba

                                                                              baba

                                                                              AB

                                                                              1

                                                                              11

                                                                              11

                                                                              111

                                                                              A B

                                                                              Review of Linear Algebra (Cont)

                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                              For matrices

                                                                              Define the Matrix Product

                                                                              (Inner Products of Rows With Columns )

                                                                              (Composition of Linear Transformations)

                                                                              Often Useful to Check Sizes

                                                                              mkk

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              nmm

                                                                              n

                                                                              bb

                                                                              bb

                                                                              B

                                                                              1

                                                                              111

                                                                              m

                                                                              iniik

                                                                              m

                                                                              iiik

                                                                              m

                                                                              inii

                                                                              m

                                                                              iii

                                                                              baba

                                                                              baba

                                                                              AB

                                                                              1

                                                                              11

                                                                              11

                                                                              111

                                                                              nmmknk

                                                                              A B

                                                                              Review of Linear Algebra (Cont)

                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                              For matrices

                                                                              Define the Matrix Product

                                                                              (Inner Products of Rows With Columns )

                                                                              (Composition of Linear Transformations)

                                                                              Often Useful to Check Sizes

                                                                              mkk

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              nmm

                                                                              n

                                                                              bb

                                                                              bb

                                                                              B

                                                                              1

                                                                              111

                                                                              m

                                                                              iniik

                                                                              m

                                                                              iiik

                                                                              m

                                                                              inii

                                                                              m

                                                                              iii

                                                                              baba

                                                                              baba

                                                                              AB

                                                                              1

                                                                              11

                                                                              11

                                                                              111

                                                                              nmmknk

                                                                              A B

                                                                              Review of Linear Algebra (Cont)

                                                                              Matrix Trace

                                                                              bull For a Square Matrix

                                                                              bull Define

                                                                              mmm

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              m

                                                                              iiiaAtr

                                                                              1)(

                                                                              Review of Linear Algebra (Cont)

                                                                              Matrix Trace

                                                                              bull For a Square Matrix

                                                                              bull Define

                                                                              bull Trace Commutes with Matrix Multiplication

                                                                              mmm

                                                                              m

                                                                              aa

                                                                              aa

                                                                              A

                                                                              1

                                                                              111

                                                                              m

                                                                              iiiaAtr

                                                                              1)(

                                                                              BAtrABtr

                                                                              Review of Linear Algebra (Cont)

                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                              bull Number of Elements in a Basis (Unique)

                                                                              Review of Linear Algebra (Cont)

                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                              bull Number of Elements in a Basis (Unique)

                                                                              bull (Use Basis Above)

                                                                              bull eg dim of a line is 1

                                                                              bull eg dim of a plane is 2

                                                                              dd dim

                                                                              Review of Linear Algebra (Cont)

                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                              bull Number of Elements in a Basis (Unique)

                                                                              bull (Use Basis Above)

                                                                              bull eg dim of a line is 1

                                                                              bull eg dim of a plane is 2

                                                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                                                              (in Statistical Uses eg ANOVA)

                                                                              dd dim

                                                                              Review of Linear Algebra (Cont)

                                                                              Norm of a Vector

                                                                              bull in d 21

                                                                              21

                                                                              1

                                                                              2 xxxx td

                                                                              jj

                                                                              Review of Linear Algebra (Cont)

                                                                              Norm of a Vector

                                                                              bull in

                                                                              bull Idea length of the vector

                                                                              d 2121

                                                                              1

                                                                              2 xxxx td

                                                                              jj

                                                                              Review of Linear Algebra (Cont)

                                                                              Norm of a Vector

                                                                              bull in

                                                                              bull Idea length of the vector

                                                                              bull Note strange properties for high

                                                                              eg ldquolength of diagonal of unit cuberdquo =

                                                                              d 2121

                                                                              1

                                                                              2 xxxx td

                                                                              jj

                                                                              d

                                                                              d

                                                                              Review of Linear Algebra (Cont)

                                                                              Norm of a Vector (cont)

                                                                              bull Length Normalized Vector

                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                              amp is a Direction Vector)

                                                                              x

                                                                              x

                                                                              Review of Linear Algebra (Cont)

                                                                              Norm of a Vector (cont)

                                                                              bull Length Normalized Vector

                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                              amp is a Direction Vector)

                                                                              bull Define Distance as

                                                                              x

                                                                              x

                                                                              yxyxyxyxd t

                                                                              Review of Linear Algebra (Cont)

                                                                              Inner (Dot Scalar) Product

                                                                              bull for Vectors and

                                                                              yxyxyx td

                                                                              jjj

                                                                              1

                                                                              x y

                                                                              Review of Linear Algebra (Cont)

                                                                              Inner (Dot Scalar) Product

                                                                              bull for Vectors and

                                                                              bull Related to Norm via

                                                                              yxyxyx td

                                                                              jjj

                                                                              1

                                                                              x y

                                                                              21 xxx

                                                                              Review of Linear Algebra (Cont)

                                                                              Inner (Dot Scalar) Product (cont)

                                                                              bull measures ldquoangle between and rdquo as

                                                                              yyxx

                                                                              yx

                                                                              yx

                                                                              yxyxangle

                                                                              tt

                                                                              t

                                                                              11 cos

                                                                              cos

                                                                              x y

                                                                              Review of Linear Algebra (Cont)

                                                                              Inner (Dot Scalar) Product (cont)

                                                                              bull measures ldquoangle between and rdquo as

                                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                                              if and only if

                                                                              yyxx

                                                                              yx

                                                                              yx

                                                                              yxyxangle

                                                                              tt

                                                                              t

                                                                              11 cos

                                                                              cos

                                                                              x y

                                                                              yx 0 yx

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis

                                                                              bull All Orthogonal to each other

                                                                              ie for

                                                                              bull All have Length 1

                                                                              ie for

                                                                              nvv 1

                                                                              1 ii vv

                                                                              0 ii vv ii

                                                                              ni 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis (cont)

                                                                              bull Spectral Representation

                                                                              where

                                                                              nvv 1

                                                                              n

                                                                              iii vax

                                                                              1

                                                                              ii vxa

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis (cont)

                                                                              bull Spectral Representation

                                                                              where

                                                                              (Coefficient is Inner Product Cool Notation)

                                                                              nvv 1

                                                                              n

                                                                              iii vax

                                                                              1

                                                                              ii vxa

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis (cont)

                                                                              bull Spectral Representation

                                                                              where

                                                                              Check

                                                                              nvv 1

                                                                              n

                                                                              iii vax

                                                                              1

                                                                              ii vxa

                                                                              iii

                                                                              n

                                                                              iii

                                                                              n

                                                                              iiii avvavvavx

                                                                              1

                                                                              1

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis (cont)

                                                                              bull Spectral Representation

                                                                              where

                                                                              Check

                                                                              bull Matrix Notation where ie

                                                                              For the Basis Matrix

                                                                              nvv 1

                                                                              n

                                                                              iii vax

                                                                              1

                                                                              ii vxa

                                                                              iii

                                                                              n

                                                                              iii

                                                                              n

                                                                              iiii avvavvavx

                                                                              1

                                                                              1

                                                                              aBx Bxa tt xBa t

                                                                              nvvB 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Orthonormal Basis (cont)

                                                                              bull Spectral Representation

                                                                              where

                                                                              Check

                                                                              bull Matrix Notation where ie

                                                                              is called transform of

                                                                              (eg Fourier or Wavelet)

                                                                              nvv 1

                                                                              n

                                                                              iii vax

                                                                              1

                                                                              ii vxa

                                                                              iii

                                                                              n

                                                                              iii

                                                                              n

                                                                              iiii avvavvavx

                                                                              1

                                                                              1

                                                                              aBx Bxa tt xBa t

                                                                              xa

                                                                              Review of Linear Algebra (Cont)

                                                                              Parseval identity for

                                                                              in subsp genrsquod by o n basis

                                                                              x

                                                                              nvv 1

                                                                              2

                                                                              1

                                                                              22

                                                                              1

                                                                              2 aavxx

                                                                              n

                                                                              ii

                                                                              n

                                                                              ii

                                                                              Review of Linear Algebra (Cont)

                                                                              Parseval identity for

                                                                              in subsp genrsquod by o n basis

                                                                              bull Pythagorean theorem

                                                                              bull ldquoDecomposition of Energyrdquo

                                                                              bull ANOVA - sums of squares

                                                                              x

                                                                              nvv 1

                                                                              2

                                                                              1

                                                                              22

                                                                              1

                                                                              2 aavxx

                                                                              n

                                                                              ii

                                                                              n

                                                                              ii

                                                                              Review of Linear Algebra (Cont)

                                                                              Parseval identity for

                                                                              in subsp genrsquod by o n basis

                                                                              bull Pythagorean theorem

                                                                              bull ldquoDecomposition of Energyrdquo

                                                                              bull ANOVA - sums of squares

                                                                              bull Transform has same length as

                                                                              ie ldquorotation in rdquo

                                                                              x

                                                                              nvv 1

                                                                              2

                                                                              1

                                                                              22

                                                                              1

                                                                              2 aavxx

                                                                              n

                                                                              ii

                                                                              n

                                                                              ii

                                                                              a xd

                                                                              Projection of a Vector onto a Subspace

                                                                              bull Idea Member of that is Closest to

                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                              Review of Linear Algebra (Cont)x

                                                                              xV

                                                                              V

                                                                              Projection of a Vector onto a Subspace

                                                                              bull Idea Member of that is Closest to

                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                              bull Find that Solves

                                                                              (ldquoLeast Squaresrdquo)

                                                                              Review of Linear Algebra (Cont)x

                                                                              xV

                                                                              V

                                                                              VxPV vxVv

                                                                              min

                                                                              Projection of a Vector onto a Subspace

                                                                              bull Idea Member of that is Closest to

                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                              bull Find that Solves

                                                                              (ldquoLeast Squaresrdquo)

                                                                              bull For Inner Product (Hilbert) Space

                                                                              Exists and is Unique

                                                                              Review of Linear Algebra (Cont)x

                                                                              xV

                                                                              V

                                                                              VxPV vxVv

                                                                              min

                                                                              xPV

                                                                              Projection of a Vector onto a Subspace (cont)

                                                                              bull General Solution in for Basis Matrix

                                                                              Review of Linear Algebra (Cont)

                                                                              d VB

                                                                              xBBBBxP tVV

                                                                              tVVV

                                                                              1

                                                                              Projection of a Vector onto a Subspace (cont)

                                                                              bull General Solution in for Basis Matrix

                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                              (thus projection is another linear operation)

                                                                              Review of Linear Algebra (Cont)

                                                                              d VB

                                                                              xBBBBxP tVV

                                                                              tVVV

                                                                              1

                                                                              tVV

                                                                              tVVV BBBBP

                                                                              1

                                                                              Projection of a Vector onto a Subspace (cont)

                                                                              bull General Solution in for Basis Matrix

                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                              (thus projection is another linear operation)

                                                                              (note same operation underlies least squares)

                                                                              Review of Linear Algebra (Cont)

                                                                              d VB

                                                                              xBBBBxP tVV

                                                                              tVVV

                                                                              1

                                                                              tVV

                                                                              tVVV BBBBP

                                                                              1

                                                                              Review of Linear Algebra (Cont)

                                                                              Projection using Orthonormal Basis

                                                                              bull Basis Matrix is Orthonormal

                                                                              nnVtV IBB

                                                                              10

                                                                              01

                                                                              1

                                                                              111

                                                                              1

                                                                              1

                                                                              nnn

                                                                              n

                                                                              ntn

                                                                              t

                                                                              vvvv

                                                                              vvvv

                                                                              vv

                                                                              v

                                                                              v

                                                                              nvv 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Projection using Orthonormal Basis

                                                                              bull Basis Matrix is Orthonormal

                                                                              bull So =

                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                              (Recall Right Multrsquon)

                                                                              nnVtV IBB

                                                                              10

                                                                              01

                                                                              1

                                                                              111

                                                                              1

                                                                              1

                                                                              nnn

                                                                              n

                                                                              ntn

                                                                              t

                                                                              vvvv

                                                                              vvvv

                                                                              vv

                                                                              v

                                                                              v

                                                                              xBBxP tVVV

                                                                              x V

                                                                              nvv 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Projection using Orthonormal Basis (cont)

                                                                              bull For Orthogonal Complement

                                                                              and

                                                                              V

                                                                              xPxPx VV 222xPxPx VV

                                                                              Review of Linear Algebra (Cont)

                                                                              Projection using Orthonormal Basis (cont)

                                                                              bull For Orthogonal Complement

                                                                              and

                                                                              bull Parseval Inequality

                                                                              V

                                                                              xPxPx VV 222xPxPx VV

                                                                              2

                                                                              1

                                                                              22

                                                                              1

                                                                              22 aavxxxP

                                                                              n

                                                                              ii

                                                                              n

                                                                              iiV

                                                                              Review of Linear Algebra (Cont)

                                                                              (Real) Unitary Matrices with

                                                                              bull Orthonormal Basis Matrix

                                                                              (So All of Above Applies)

                                                                              ddU IUU t

                                                                              Review of Linear Algebra (Cont)

                                                                              (Real) Unitary Matrices with

                                                                              bull Orthonormal Basis Matrix

                                                                              (So All of Above Applies)

                                                                              bull Note Transformrsquon is Distance Preserving

                                                                              ddU IUU t

                                                                              yxdyxyxyxUyUxUdn

                                                                              i ii 2

                                                                              1

                                                                              Review of Linear Algebra (Cont)

                                                                              (Real) Unitary Matrices with

                                                                              bull Orthonormal Basis Matrix

                                                                              (So All of Above Applies)

                                                                              bull Note Transformrsquon is Distance Preserving

                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                              ddU IUU t

                                                                              yxdyxyxyxUyUxUdn

                                                                              i ii 2

                                                                              1

                                                                              U

                                                                              Review of Linear Algebra (Cont)

                                                                              Singular Value Decomposition (SVD)

                                                                              For a Matrix

                                                                              Find

                                                                              ndX

                                                                              Review of Linear Algebra (Cont)

                                                                              Singular Value Decomposition (SVD)

                                                                              For a Matrix

                                                                              Find a Diagonal Matrix

                                                                              with Entries

                                                                              called Singular Values

                                                                              ndX

                                                                              ndS

                                                                              )min(1 ndss

                                                                              Review of Linear Algebra (Cont)

                                                                              Singular Value Decomposition (SVD)

                                                                              For a Matrix

                                                                              Find a Diagonal Matrix

                                                                              with Entries

                                                                              called Singular Values

                                                                              And Unitary (Rotation) Matrices

                                                                              (recall )

                                                                              ndX

                                                                              ndS

                                                                              )min(1 ndss

                                                                              ddU nnV

                                                                              IVVUU tt

                                                                              Review of Linear Algebra (Cont)

                                                                              Singular Value Decomposition (SVD)

                                                                              For a Matrix

                                                                              Find a Diagonal Matrix

                                                                              with Entries

                                                                              called Singular Values

                                                                              And Unitary (Rotation) Matrices

                                                                              (recall )

                                                                              So That

                                                                              ndX

                                                                              ndS

                                                                              )min(1 ndss

                                                                              ddU nnV

                                                                              IVVUU tt tUSVX

                                                                              Review of Linear Algebra (Cont)

                                                                              Intuition behind Singular Value Decomposition

                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                              vVSUvVSUvX tt

                                                                              Review of Linear Algebra (Cont)

                                                                              Intuition behind Singular Value Decomposition

                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                              bull First rotate

                                                                              X

                                                                              vVSUvVSUvX tt

                                                                              Review of Linear Algebra (Cont)

                                                                              Intuition behind Singular Value Decomposition

                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                              bull First rotate

                                                                              bull Second rescale coordinate axes (by )

                                                                              X

                                                                              vVSUvVSUvX tt

                                                                              is

                                                                              Review of Linear Algebra (Cont)

                                                                              Intuition behind Singular Value Decomposition

                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                              bull First rotate

                                                                              bull Second rescale coordinate axes (by )

                                                                              bull Third rotate again

                                                                              X

                                                                              vVSUvVSUvX tt

                                                                              is

                                                                              Review of Linear Algebra (Cont)

                                                                              Intuition behind Singular Value Decomposition

                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                              bull First rotate

                                                                              bull Second rescale coordinate axes (by )

                                                                              bull Third rotate again

                                                                              bull ie have diagonalized the transformation

                                                                              X

                                                                              vVSUvVSUvX tt

                                                                              is

                                                                              SVD Compact Representation

                                                                              Useful Labeling

                                                                              Singular Values in Increasing Order

                                                                              Review of Linear Algebra (Cont)

                                                                              )min(1 dnss

                                                                              SVD Compact Representation

                                                                              Useful Labeling

                                                                              Singular Values in Increasing Order

                                                                              Note singular values = 0 can be omitted

                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                              Review of Linear Algebra (Cont)

                                                                              )min(1 dnss

                                                                              Review of Linear Algebra (Cont)

                                                                              r

                                                                              SVD Compact Representation

                                                                              Useful Labeling

                                                                              Singular Values in Increasing Order

                                                                              Note singular values = 0 can be omitted

                                                                              Let = of positive singular values

                                                                              )min(1 dnss

                                                                              Review of Linear Algebra (Cont)

                                                                              r

                                                                              SVD Compact Representation

                                                                              Useful Labeling

                                                                              Singular Values in Increasing Order

                                                                              Note singular values = 0 can be omitted

                                                                              Let = of positive singular values

                                                                              Then

                                                                              Where are truncations of

                                                                              trnrrrd VSUX

                                                                              VSU

                                                                              )min(1 dnss

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Full Representation

                                                                              =

                                                                              Graphics Display Assumes

                                                                              ndX ddU ndS nn

                                                                              tV

                                                                              nd

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Full Representation

                                                                              =

                                                                              Full Rank Basis Matrix

                                                                              ndX ddU ndS nn

                                                                              tV

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Full Representation

                                                                              =

                                                                              Full Rank Basis Matrix

                                                                              All 0s in Bottom

                                                                              ndX ddU ndS nn

                                                                              tV

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Reduced Representation

                                                                              =

                                                                              These Columns Get 0ed Out

                                                                              ndX ddU nnS nn

                                                                              tV

                                                                              nnd 0

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Reduced Representation

                                                                              =ndX ndU nnS nn

                                                                              tV

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Reduced Representation

                                                                              =

                                                                              Also Some of These May be 0

                                                                              ndX ndU nnS nn

                                                                              tV

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Compact Representation

                                                                              = ndX rdU

                                                                              rrS nrtV

                                                                              0

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Compact Representation

                                                                              =

                                                                              These Get 0ed Out

                                                                              ndX rdU

                                                                              rrS nrtV

                                                                              0

                                                                              Review of Linear Algebra (Cont)

                                                                              SVD Compact Representation

                                                                              = ndX rdU

                                                                              rrS nrtV

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition

                                                                              For a (Symmetric) Square Matrix

                                                                              Find

                                                                              ddX

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition

                                                                              For a (Symmetric) Square Matrix

                                                                              Find a Diagonal Matrix

                                                                              Called Eigenvalues

                                                                              Convenient Ordering

                                                                              ddX

                                                                              d

                                                                              D

                                                                              0

                                                                              01

                                                                              n 1

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition

                                                                              For a (Symmetric) Square Matrix

                                                                              Find a Diagonal Matrix

                                                                              And an Orthonormal Matrix

                                                                              (ie )

                                                                              ddX

                                                                              d

                                                                              D

                                                                              0

                                                                              01

                                                                              ddB

                                                                              ddtt IBBBB

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition

                                                                              For a (Symmetric) Square Matrix

                                                                              Find a Diagonal Matrix

                                                                              And an Orthonormal Matrix

                                                                              (ie )

                                                                              So that ie

                                                                              ddX

                                                                              d

                                                                              D

                                                                              0

                                                                              01

                                                                              ddB

                                                                              ddtt IBBBB

                                                                              DBBX tBDBX

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                              (looks similar)

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                              Complex (uses )

                                                                              VU

                                                                              1i

                                                                              Review of Linear Algebra (Cont)

                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                              VU

                                                                              X

                                                                              BVU

                                                                              1i

                                                                              • Participant Presentations
                                                                              • PCA to find clusters
                                                                              • Statistical Smoothing
                                                                              • Kernel Density Estimation
                                                                              • Scatterplot Smoothing
                                                                              • Statistical Smoothing (2)
                                                                              • SiZer Background
                                                                              • SiZer Background (2)
                                                                              • SiZer Background (3)
                                                                              • SiZer Background (4)
                                                                              • SiZer Background (5)
                                                                              • SiZer Overview
                                                                              • PCA to find clusters (2)
                                                                              • PCA to find clusters (3)
                                                                              • PCA to find clusters (4)
                                                                              • PCA to find clusters (5)
                                                                              • PCA to find clusters (6)
                                                                              • PCA to find clusters (7)
                                                                              • Recall Yeast Cell Cycle Data
                                                                              • Yeast Cell Cycle Data FDA View
                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                              • Frequency 2 Analysis
                                                                              • Frequency 2 Analysis (2)
                                                                              • Frequency 2 Analysis (3)
                                                                              • Yeast Cell Cycle
                                                                              • SiZer Study of Distrsquon of Angles
                                                                              • Reclassification of Major Genes
                                                                              • Compare to Previous Classifrsquon
                                                                              • New Subpopulation View
                                                                              • New Subpopulation View (2)
                                                                              • Detailed Look at PCA
                                                                              • PCA Rediscovery ndash Renaming
                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                              • An Interesting Historical Note
                                                                              • An Interesting Historical Note (2)
                                                                              • Detailed Look at PCA (2)
                                                                              • Detailed Look at PCA (3)
                                                                              • Review of Linear Algebra
                                                                              • Review of Linear Algebra (2)
                                                                              • Review of Linear Algebra (3)
                                                                              • Review of Linear Algebra (Cont)
                                                                              • Review of Linear Algebra (Cont) (2)
                                                                              • Review of Linear Algebra (Cont) (3)
                                                                              • Review of Linear Algebra (Cont) (4)
                                                                              • Review of Linear Algebra (Cont) (5)
                                                                              • Review of Linear Algebra (Cont) (6)
                                                                              • Review of Linear Algebra (Cont) (7)
                                                                              • Review of Linear Algebra (Cont) (8)
                                                                              • Review of Linear Algebra (Cont) (9)
                                                                              • Review of Linear Algebra (Cont) (10)
                                                                              • Review of Linear Algebra (Cont) (11)
                                                                              • Review of Linear Algebra (Cont) (12)
                                                                              • Review of Linear Algebra (Cont) (13)
                                                                              • Review of Linear Algebra (Cont) (14)
                                                                              • Review of Linear Algebra (Cont) (15)
                                                                              • Review of Linear Algebra (Cont) (16)
                                                                              • Review of Linear Algebra (Cont) (17)
                                                                              • Review of Linear Algebra (Cont) (18)
                                                                              • Review of Linear Algebra (Cont) (19)
                                                                              • Review of Linear Algebra (Cont) (20)
                                                                              • Review of Linear Algebra (Cont) (21)
                                                                              • Review of Linear Algebra (Cont) (22)
                                                                              • Review of Linear Algebra (Cont) (23)
                                                                              • Review of Linear Algebra (Cont) (24)
                                                                              • Review of Linear Algebra (Cont) (25)
                                                                              • Review of Linear Algebra (Cont) (26)
                                                                              • Review of Linear Algebra (Cont) (27)
                                                                              • Review of Linear Algebra (Cont) (28)
                                                                              • Review of Linear Algebra (Cont) (29)
                                                                              • Review of Linear Algebra (Cont) (30)
                                                                              • Review of Linear Algebra (Cont) (31)
                                                                              • Review of Linear Algebra (Cont) (32)
                                                                              • Review of Linear Algebra (Cont) (33)
                                                                              • Review of Linear Algebra (Cont) (34)
                                                                              • Review of Linear Algebra (Cont) (35)
                                                                              • Review of Linear Algebra (Cont) (36)
                                                                              • Review of Linear Algebra (Cont) (37)
                                                                              • Review of Linear Algebra (Cont) (38)
                                                                              • Review of Linear Algebra (Cont) (39)
                                                                              • Review of Linear Algebra (Cont) (40)
                                                                              • Review of Linear Algebra (Cont) (41)
                                                                              • Review of Linear Algebra (Cont) (42)
                                                                              • Review of Linear Algebra (Cont) (43)
                                                                              • Review of Linear Algebra (Cont) (44)
                                                                              • Review of Linear Algebra (Cont) (45)
                                                                              • Review of Linear Algebra (Cont) (46)
                                                                              • Review of Linear Algebra (Cont) (47)
                                                                              • Review of Linear Algebra (Cont) (48)
                                                                              • Review of Linear Algebra (Cont) (49)
                                                                              • Review of Linear Algebra (Cont) (50)
                                                                              • Review of Linear Algebra (Cont) (51)
                                                                              • Review of Linear Algebra (Cont) (52)
                                                                              • Review of Linear Algebra (Cont) (53)
                                                                              • Review of Linear Algebra (Cont) (54)
                                                                              • Review of Linear Algebra (Cont) (55)
                                                                              • Review of Linear Algebra (Cont) (56)
                                                                              • Review of Linear Algebra (Cont) (57)
                                                                              • Review of Linear Algebra (Cont) (58)
                                                                              • Review of Linear Algebra (Cont) (59)
                                                                              • Review of Linear Algebra (Cont) (60)
                                                                              • Review of Linear Algebra (Cont) (61)
                                                                              • Review of Linear Algebra (Cont) (62)
                                                                              • Review of Linear Algebra (Cont) (63)
                                                                              • Review of Linear Algebra (Cont) (64)
                                                                              • Review of Linear Algebra (Cont) (65)
                                                                              • Review of Linear Algebra (Cont) (66)
                                                                              • Review of Linear Algebra (Cont) (67)
                                                                              • Review of Linear Algebra (Cont) (68)
                                                                              • Review of Linear Algebra (Cont) (69)
                                                                              • Review of Linear Algebra (Cont) (70)
                                                                              • Review of Linear Algebra (Cont) (71)
                                                                              • Review of Linear Algebra (Cont) (72)
                                                                              • Review of Linear Algebra (Cont) (73)
                                                                              • Review of Linear Algebra (Cont) (74)
                                                                              • Review of Linear Algebra (Cont) (75)
                                                                              • Review of Linear Algebra (Cont) (76)
                                                                              • Review of Linear Algebra (Cont) (77)
                                                                              • Review of Linear Algebra (Cont) (78)
                                                                              • Review of Linear Algebra (Cont) (79)
                                                                              • Review of Linear Algebra (Cont) (80)
                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                An Interesting Historical Note

                                                                                The 1st () application of PCA to Functional

                                                                                Data Analysis

                                                                                Rao (1958)

                                                                                1st Paper with ldquoCurves as Data Objectsrdquo

                                                                                viewpoint

                                                                                Detailed Look at PCA

                                                                                Three Important (amp Interesting) Viewpoints

                                                                                1 Mathematics

                                                                                2 Numerics

                                                                                3 Statistics

                                                                                Goal Study Interrelationships

                                                                                Detailed Look at PCA

                                                                                Three Important (amp Interesting) Viewpoints

                                                                                1 Mathematics

                                                                                2 Numerics

                                                                                3 Statistics

                                                                                1st Review Linear Alg and Multivar Prob

                                                                                Review of Linear Algebra

                                                                                Vector Space

                                                                                bull set of ldquovectorsrdquo

                                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                                xa

                                                                                Review of Linear Algebra

                                                                                Vector Space

                                                                                bull set of ldquovectorsrdquo

                                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                ( in space)

                                                                                xa

                                                                                i

                                                                                ii xa

                                                                                Review of Linear Algebra

                                                                                Vector Space

                                                                                bull set of ldquovectorsrdquo

                                                                                bull and ldquoscalarsrdquo (coefficients)

                                                                                bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                ( in space)

                                                                                eg

                                                                                ldquo dim Euclidrsquon spacerdquo

                                                                                xa

                                                                                i

                                                                                ii xa

                                                                                d

                                                                                d

                                                                                d xx

                                                                                x

                                                                                x

                                                                                x 1

                                                                                1

                                                                                d

                                                                                Review of Linear Algebra (Cont)

                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                Review of Linear Algebra (Cont)

                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                Review of Linear Algebra (Cont)

                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                Note Planes not Through the Origin

                                                                                are not Subspaces

                                                                                (Do not Contain )00 x

                                                                                Review of Linear Algebra (Cont)

                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                (all Linear Combos of them =

                                                                                = Containing Hyperplane

                                                                                through Origin)

                                                                                Review of Linear Algebra (Cont)

                                                                                Basis of Subspace Set of Vectors that

                                                                                bull Span ie Everything is a Lin Com of them

                                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                Review of Linear Algebra (Cont)

                                                                                Basis of Subspace Set of Vectors that

                                                                                bull Span ie Everything is a Lin Com of them

                                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                bull eg ldquoUnit Vector Basisrdquo

                                                                                bull Since

                                                                                d

                                                                                1

                                                                                0

                                                                                0

                                                                                0

                                                                                1

                                                                                0

                                                                                0

                                                                                0

                                                                                1

                                                                                1

                                                                                0

                                                                                0

                                                                                0

                                                                                1

                                                                                0

                                                                                0

                                                                                0

                                                                                1

                                                                                212

                                                                                1

                                                                                d

                                                                                d

                                                                                xxx

                                                                                x

                                                                                x

                                                                                x

                                                                                Review of Linear Algebra (Cont)

                                                                                Basis Matrix of subspace of

                                                                                Given a basis

                                                                                create matrix of columns

                                                                                dnvv 1

                                                                                nddnd

                                                                                n

                                                                                n

                                                                                vv

                                                                                vv

                                                                                vvB

                                                                                1

                                                                                111

                                                                                1

                                                                                Review of Linear Algebra (Cont)

                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                where

                                                                                n

                                                                                iii aBva

                                                                                1

                                                                                na

                                                                                a

                                                                                a 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                where

                                                                                Note Right Multiplication Gives

                                                                                Linear Combination of Column Vectors

                                                                                n

                                                                                iii aBva

                                                                                1

                                                                                na

                                                                                a

                                                                                a 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                where

                                                                                Check sizes

                                                                                n

                                                                                iii aBva

                                                                                1

                                                                                na

                                                                                a

                                                                                a 1

                                                                                )1()(1 nndd

                                                                                Review of Linear Algebra (Cont)

                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                For matrices

                                                                                Define the Matrix Product

                                                                                mkk

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                nmm

                                                                                n

                                                                                bb

                                                                                bb

                                                                                B

                                                                                1

                                                                                111

                                                                                m

                                                                                iniik

                                                                                m

                                                                                iiik

                                                                                m

                                                                                inii

                                                                                m

                                                                                iii

                                                                                baba

                                                                                baba

                                                                                AB

                                                                                1

                                                                                11

                                                                                11

                                                                                111

                                                                                Review of Linear Algebra (Cont)

                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                For matrices

                                                                                Define the Matrix Product

                                                                                (Inner Products of Rows With Columns )

                                                                                mkk

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                nmm

                                                                                n

                                                                                bb

                                                                                bb

                                                                                B

                                                                                1

                                                                                111

                                                                                m

                                                                                iniik

                                                                                m

                                                                                iiik

                                                                                m

                                                                                inii

                                                                                m

                                                                                iii

                                                                                baba

                                                                                baba

                                                                                AB

                                                                                1

                                                                                11

                                                                                11

                                                                                111

                                                                                A B

                                                                                Review of Linear Algebra (Cont)

                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                For matrices

                                                                                Define the Matrix Product

                                                                                (Inner Products of Rows With Columns )

                                                                                (Composition of Linear Transformations)

                                                                                mkk

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                nmm

                                                                                n

                                                                                bb

                                                                                bb

                                                                                B

                                                                                1

                                                                                111

                                                                                m

                                                                                iniik

                                                                                m

                                                                                iiik

                                                                                m

                                                                                inii

                                                                                m

                                                                                iii

                                                                                baba

                                                                                baba

                                                                                AB

                                                                                1

                                                                                11

                                                                                11

                                                                                111

                                                                                A B

                                                                                Review of Linear Algebra (Cont)

                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                For matrices

                                                                                Define the Matrix Product

                                                                                (Inner Products of Rows With Columns )

                                                                                (Composition of Linear Transformations)

                                                                                Often Useful to Check Sizes

                                                                                mkk

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                nmm

                                                                                n

                                                                                bb

                                                                                bb

                                                                                B

                                                                                1

                                                                                111

                                                                                m

                                                                                iniik

                                                                                m

                                                                                iiik

                                                                                m

                                                                                inii

                                                                                m

                                                                                iii

                                                                                baba

                                                                                baba

                                                                                AB

                                                                                1

                                                                                11

                                                                                11

                                                                                111

                                                                                nmmknk

                                                                                A B

                                                                                Review of Linear Algebra (Cont)

                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                For matrices

                                                                                Define the Matrix Product

                                                                                (Inner Products of Rows With Columns )

                                                                                (Composition of Linear Transformations)

                                                                                Often Useful to Check Sizes

                                                                                mkk

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                nmm

                                                                                n

                                                                                bb

                                                                                bb

                                                                                B

                                                                                1

                                                                                111

                                                                                m

                                                                                iniik

                                                                                m

                                                                                iiik

                                                                                m

                                                                                inii

                                                                                m

                                                                                iii

                                                                                baba

                                                                                baba

                                                                                AB

                                                                                1

                                                                                11

                                                                                11

                                                                                111

                                                                                nmmknk

                                                                                A B

                                                                                Review of Linear Algebra (Cont)

                                                                                Matrix Trace

                                                                                bull For a Square Matrix

                                                                                bull Define

                                                                                mmm

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                m

                                                                                iiiaAtr

                                                                                1)(

                                                                                Review of Linear Algebra (Cont)

                                                                                Matrix Trace

                                                                                bull For a Square Matrix

                                                                                bull Define

                                                                                bull Trace Commutes with Matrix Multiplication

                                                                                mmm

                                                                                m

                                                                                aa

                                                                                aa

                                                                                A

                                                                                1

                                                                                111

                                                                                m

                                                                                iiiaAtr

                                                                                1)(

                                                                                BAtrABtr

                                                                                Review of Linear Algebra (Cont)

                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                bull Number of Elements in a Basis (Unique)

                                                                                Review of Linear Algebra (Cont)

                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                bull Number of Elements in a Basis (Unique)

                                                                                bull (Use Basis Above)

                                                                                bull eg dim of a line is 1

                                                                                bull eg dim of a plane is 2

                                                                                dd dim

                                                                                Review of Linear Algebra (Cont)

                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                bull Number of Elements in a Basis (Unique)

                                                                                bull (Use Basis Above)

                                                                                bull eg dim of a line is 1

                                                                                bull eg dim of a plane is 2

                                                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                (in Statistical Uses eg ANOVA)

                                                                                dd dim

                                                                                Review of Linear Algebra (Cont)

                                                                                Norm of a Vector

                                                                                bull in d 21

                                                                                21

                                                                                1

                                                                                2 xxxx td

                                                                                jj

                                                                                Review of Linear Algebra (Cont)

                                                                                Norm of a Vector

                                                                                bull in

                                                                                bull Idea length of the vector

                                                                                d 2121

                                                                                1

                                                                                2 xxxx td

                                                                                jj

                                                                                Review of Linear Algebra (Cont)

                                                                                Norm of a Vector

                                                                                bull in

                                                                                bull Idea length of the vector

                                                                                bull Note strange properties for high

                                                                                eg ldquolength of diagonal of unit cuberdquo =

                                                                                d 2121

                                                                                1

                                                                                2 xxxx td

                                                                                jj

                                                                                d

                                                                                d

                                                                                Review of Linear Algebra (Cont)

                                                                                Norm of a Vector (cont)

                                                                                bull Length Normalized Vector

                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                amp is a Direction Vector)

                                                                                x

                                                                                x

                                                                                Review of Linear Algebra (Cont)

                                                                                Norm of a Vector (cont)

                                                                                bull Length Normalized Vector

                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                amp is a Direction Vector)

                                                                                bull Define Distance as

                                                                                x

                                                                                x

                                                                                yxyxyxyxd t

                                                                                Review of Linear Algebra (Cont)

                                                                                Inner (Dot Scalar) Product

                                                                                bull for Vectors and

                                                                                yxyxyx td

                                                                                jjj

                                                                                1

                                                                                x y

                                                                                Review of Linear Algebra (Cont)

                                                                                Inner (Dot Scalar) Product

                                                                                bull for Vectors and

                                                                                bull Related to Norm via

                                                                                yxyxyx td

                                                                                jjj

                                                                                1

                                                                                x y

                                                                                21 xxx

                                                                                Review of Linear Algebra (Cont)

                                                                                Inner (Dot Scalar) Product (cont)

                                                                                bull measures ldquoangle between and rdquo as

                                                                                yyxx

                                                                                yx

                                                                                yx

                                                                                yxyxangle

                                                                                tt

                                                                                t

                                                                                11 cos

                                                                                cos

                                                                                x y

                                                                                Review of Linear Algebra (Cont)

                                                                                Inner (Dot Scalar) Product (cont)

                                                                                bull measures ldquoangle between and rdquo as

                                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                                if and only if

                                                                                yyxx

                                                                                yx

                                                                                yx

                                                                                yxyxangle

                                                                                tt

                                                                                t

                                                                                11 cos

                                                                                cos

                                                                                x y

                                                                                yx 0 yx

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis

                                                                                bull All Orthogonal to each other

                                                                                ie for

                                                                                bull All have Length 1

                                                                                ie for

                                                                                nvv 1

                                                                                1 ii vv

                                                                                0 ii vv ii

                                                                                ni 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis (cont)

                                                                                bull Spectral Representation

                                                                                where

                                                                                nvv 1

                                                                                n

                                                                                iii vax

                                                                                1

                                                                                ii vxa

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis (cont)

                                                                                bull Spectral Representation

                                                                                where

                                                                                (Coefficient is Inner Product Cool Notation)

                                                                                nvv 1

                                                                                n

                                                                                iii vax

                                                                                1

                                                                                ii vxa

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis (cont)

                                                                                bull Spectral Representation

                                                                                where

                                                                                Check

                                                                                nvv 1

                                                                                n

                                                                                iii vax

                                                                                1

                                                                                ii vxa

                                                                                iii

                                                                                n

                                                                                iii

                                                                                n

                                                                                iiii avvavvavx

                                                                                1

                                                                                1

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis (cont)

                                                                                bull Spectral Representation

                                                                                where

                                                                                Check

                                                                                bull Matrix Notation where ie

                                                                                For the Basis Matrix

                                                                                nvv 1

                                                                                n

                                                                                iii vax

                                                                                1

                                                                                ii vxa

                                                                                iii

                                                                                n

                                                                                iii

                                                                                n

                                                                                iiii avvavvavx

                                                                                1

                                                                                1

                                                                                aBx Bxa tt xBa t

                                                                                nvvB 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Orthonormal Basis (cont)

                                                                                bull Spectral Representation

                                                                                where

                                                                                Check

                                                                                bull Matrix Notation where ie

                                                                                is called transform of

                                                                                (eg Fourier or Wavelet)

                                                                                nvv 1

                                                                                n

                                                                                iii vax

                                                                                1

                                                                                ii vxa

                                                                                iii

                                                                                n

                                                                                iii

                                                                                n

                                                                                iiii avvavvavx

                                                                                1

                                                                                1

                                                                                aBx Bxa tt xBa t

                                                                                xa

                                                                                Review of Linear Algebra (Cont)

                                                                                Parseval identity for

                                                                                in subsp genrsquod by o n basis

                                                                                x

                                                                                nvv 1

                                                                                2

                                                                                1

                                                                                22

                                                                                1

                                                                                2 aavxx

                                                                                n

                                                                                ii

                                                                                n

                                                                                ii

                                                                                Review of Linear Algebra (Cont)

                                                                                Parseval identity for

                                                                                in subsp genrsquod by o n basis

                                                                                bull Pythagorean theorem

                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                bull ANOVA - sums of squares

                                                                                x

                                                                                nvv 1

                                                                                2

                                                                                1

                                                                                22

                                                                                1

                                                                                2 aavxx

                                                                                n

                                                                                ii

                                                                                n

                                                                                ii

                                                                                Review of Linear Algebra (Cont)

                                                                                Parseval identity for

                                                                                in subsp genrsquod by o n basis

                                                                                bull Pythagorean theorem

                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                bull ANOVA - sums of squares

                                                                                bull Transform has same length as

                                                                                ie ldquorotation in rdquo

                                                                                x

                                                                                nvv 1

                                                                                2

                                                                                1

                                                                                22

                                                                                1

                                                                                2 aavxx

                                                                                n

                                                                                ii

                                                                                n

                                                                                ii

                                                                                a xd

                                                                                Projection of a Vector onto a Subspace

                                                                                bull Idea Member of that is Closest to

                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                Review of Linear Algebra (Cont)x

                                                                                xV

                                                                                V

                                                                                Projection of a Vector onto a Subspace

                                                                                bull Idea Member of that is Closest to

                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                bull Find that Solves

                                                                                (ldquoLeast Squaresrdquo)

                                                                                Review of Linear Algebra (Cont)x

                                                                                xV

                                                                                V

                                                                                VxPV vxVv

                                                                                min

                                                                                Projection of a Vector onto a Subspace

                                                                                bull Idea Member of that is Closest to

                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                bull Find that Solves

                                                                                (ldquoLeast Squaresrdquo)

                                                                                bull For Inner Product (Hilbert) Space

                                                                                Exists and is Unique

                                                                                Review of Linear Algebra (Cont)x

                                                                                xV

                                                                                V

                                                                                VxPV vxVv

                                                                                min

                                                                                xPV

                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                bull General Solution in for Basis Matrix

                                                                                Review of Linear Algebra (Cont)

                                                                                d VB

                                                                                xBBBBxP tVV

                                                                                tVVV

                                                                                1

                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                bull General Solution in for Basis Matrix

                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                (thus projection is another linear operation)

                                                                                Review of Linear Algebra (Cont)

                                                                                d VB

                                                                                xBBBBxP tVV

                                                                                tVVV

                                                                                1

                                                                                tVV

                                                                                tVVV BBBBP

                                                                                1

                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                bull General Solution in for Basis Matrix

                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                (thus projection is another linear operation)

                                                                                (note same operation underlies least squares)

                                                                                Review of Linear Algebra (Cont)

                                                                                d VB

                                                                                xBBBBxP tVV

                                                                                tVVV

                                                                                1

                                                                                tVV

                                                                                tVVV BBBBP

                                                                                1

                                                                                Review of Linear Algebra (Cont)

                                                                                Projection using Orthonormal Basis

                                                                                bull Basis Matrix is Orthonormal

                                                                                nnVtV IBB

                                                                                10

                                                                                01

                                                                                1

                                                                                111

                                                                                1

                                                                                1

                                                                                nnn

                                                                                n

                                                                                ntn

                                                                                t

                                                                                vvvv

                                                                                vvvv

                                                                                vv

                                                                                v

                                                                                v

                                                                                nvv 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Projection using Orthonormal Basis

                                                                                bull Basis Matrix is Orthonormal

                                                                                bull So =

                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                (Recall Right Multrsquon)

                                                                                nnVtV IBB

                                                                                10

                                                                                01

                                                                                1

                                                                                111

                                                                                1

                                                                                1

                                                                                nnn

                                                                                n

                                                                                ntn

                                                                                t

                                                                                vvvv

                                                                                vvvv

                                                                                vv

                                                                                v

                                                                                v

                                                                                xBBxP tVVV

                                                                                x V

                                                                                nvv 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Projection using Orthonormal Basis (cont)

                                                                                bull For Orthogonal Complement

                                                                                and

                                                                                V

                                                                                xPxPx VV 222xPxPx VV

                                                                                Review of Linear Algebra (Cont)

                                                                                Projection using Orthonormal Basis (cont)

                                                                                bull For Orthogonal Complement

                                                                                and

                                                                                bull Parseval Inequality

                                                                                V

                                                                                xPxPx VV 222xPxPx VV

                                                                                2

                                                                                1

                                                                                22

                                                                                1

                                                                                22 aavxxxP

                                                                                n

                                                                                ii

                                                                                n

                                                                                iiV

                                                                                Review of Linear Algebra (Cont)

                                                                                (Real) Unitary Matrices with

                                                                                bull Orthonormal Basis Matrix

                                                                                (So All of Above Applies)

                                                                                ddU IUU t

                                                                                Review of Linear Algebra (Cont)

                                                                                (Real) Unitary Matrices with

                                                                                bull Orthonormal Basis Matrix

                                                                                (So All of Above Applies)

                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                ddU IUU t

                                                                                yxdyxyxyxUyUxUdn

                                                                                i ii 2

                                                                                1

                                                                                Review of Linear Algebra (Cont)

                                                                                (Real) Unitary Matrices with

                                                                                bull Orthonormal Basis Matrix

                                                                                (So All of Above Applies)

                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                ddU IUU t

                                                                                yxdyxyxyxUyUxUdn

                                                                                i ii 2

                                                                                1

                                                                                U

                                                                                Review of Linear Algebra (Cont)

                                                                                Singular Value Decomposition (SVD)

                                                                                For a Matrix

                                                                                Find

                                                                                ndX

                                                                                Review of Linear Algebra (Cont)

                                                                                Singular Value Decomposition (SVD)

                                                                                For a Matrix

                                                                                Find a Diagonal Matrix

                                                                                with Entries

                                                                                called Singular Values

                                                                                ndX

                                                                                ndS

                                                                                )min(1 ndss

                                                                                Review of Linear Algebra (Cont)

                                                                                Singular Value Decomposition (SVD)

                                                                                For a Matrix

                                                                                Find a Diagonal Matrix

                                                                                with Entries

                                                                                called Singular Values

                                                                                And Unitary (Rotation) Matrices

                                                                                (recall )

                                                                                ndX

                                                                                ndS

                                                                                )min(1 ndss

                                                                                ddU nnV

                                                                                IVVUU tt

                                                                                Review of Linear Algebra (Cont)

                                                                                Singular Value Decomposition (SVD)

                                                                                For a Matrix

                                                                                Find a Diagonal Matrix

                                                                                with Entries

                                                                                called Singular Values

                                                                                And Unitary (Rotation) Matrices

                                                                                (recall )

                                                                                So That

                                                                                ndX

                                                                                ndS

                                                                                )min(1 ndss

                                                                                ddU nnV

                                                                                IVVUU tt tUSVX

                                                                                Review of Linear Algebra (Cont)

                                                                                Intuition behind Singular Value Decomposition

                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                vVSUvVSUvX tt

                                                                                Review of Linear Algebra (Cont)

                                                                                Intuition behind Singular Value Decomposition

                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                bull First rotate

                                                                                X

                                                                                vVSUvVSUvX tt

                                                                                Review of Linear Algebra (Cont)

                                                                                Intuition behind Singular Value Decomposition

                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                bull First rotate

                                                                                bull Second rescale coordinate axes (by )

                                                                                X

                                                                                vVSUvVSUvX tt

                                                                                is

                                                                                Review of Linear Algebra (Cont)

                                                                                Intuition behind Singular Value Decomposition

                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                bull First rotate

                                                                                bull Second rescale coordinate axes (by )

                                                                                bull Third rotate again

                                                                                X

                                                                                vVSUvVSUvX tt

                                                                                is

                                                                                Review of Linear Algebra (Cont)

                                                                                Intuition behind Singular Value Decomposition

                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                bull First rotate

                                                                                bull Second rescale coordinate axes (by )

                                                                                bull Third rotate again

                                                                                bull ie have diagonalized the transformation

                                                                                X

                                                                                vVSUvVSUvX tt

                                                                                is

                                                                                SVD Compact Representation

                                                                                Useful Labeling

                                                                                Singular Values in Increasing Order

                                                                                Review of Linear Algebra (Cont)

                                                                                )min(1 dnss

                                                                                SVD Compact Representation

                                                                                Useful Labeling

                                                                                Singular Values in Increasing Order

                                                                                Note singular values = 0 can be omitted

                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                Review of Linear Algebra (Cont)

                                                                                )min(1 dnss

                                                                                Review of Linear Algebra (Cont)

                                                                                r

                                                                                SVD Compact Representation

                                                                                Useful Labeling

                                                                                Singular Values in Increasing Order

                                                                                Note singular values = 0 can be omitted

                                                                                Let = of positive singular values

                                                                                )min(1 dnss

                                                                                Review of Linear Algebra (Cont)

                                                                                r

                                                                                SVD Compact Representation

                                                                                Useful Labeling

                                                                                Singular Values in Increasing Order

                                                                                Note singular values = 0 can be omitted

                                                                                Let = of positive singular values

                                                                                Then

                                                                                Where are truncations of

                                                                                trnrrrd VSUX

                                                                                VSU

                                                                                )min(1 dnss

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Full Representation

                                                                                =

                                                                                Graphics Display Assumes

                                                                                ndX ddU ndS nn

                                                                                tV

                                                                                nd

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Full Representation

                                                                                =

                                                                                Full Rank Basis Matrix

                                                                                ndX ddU ndS nn

                                                                                tV

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Full Representation

                                                                                =

                                                                                Full Rank Basis Matrix

                                                                                All 0s in Bottom

                                                                                ndX ddU ndS nn

                                                                                tV

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Reduced Representation

                                                                                =

                                                                                These Columns Get 0ed Out

                                                                                ndX ddU nnS nn

                                                                                tV

                                                                                nnd 0

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Reduced Representation

                                                                                =ndX ndU nnS nn

                                                                                tV

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Reduced Representation

                                                                                =

                                                                                Also Some of These May be 0

                                                                                ndX ndU nnS nn

                                                                                tV

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Compact Representation

                                                                                = ndX rdU

                                                                                rrS nrtV

                                                                                0

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Compact Representation

                                                                                =

                                                                                These Get 0ed Out

                                                                                ndX rdU

                                                                                rrS nrtV

                                                                                0

                                                                                Review of Linear Algebra (Cont)

                                                                                SVD Compact Representation

                                                                                = ndX rdU

                                                                                rrS nrtV

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition

                                                                                For a (Symmetric) Square Matrix

                                                                                Find

                                                                                ddX

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition

                                                                                For a (Symmetric) Square Matrix

                                                                                Find a Diagonal Matrix

                                                                                Called Eigenvalues

                                                                                Convenient Ordering

                                                                                ddX

                                                                                d

                                                                                D

                                                                                0

                                                                                01

                                                                                n 1

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition

                                                                                For a (Symmetric) Square Matrix

                                                                                Find a Diagonal Matrix

                                                                                And an Orthonormal Matrix

                                                                                (ie )

                                                                                ddX

                                                                                d

                                                                                D

                                                                                0

                                                                                01

                                                                                ddB

                                                                                ddtt IBBBB

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition

                                                                                For a (Symmetric) Square Matrix

                                                                                Find a Diagonal Matrix

                                                                                And an Orthonormal Matrix

                                                                                (ie )

                                                                                So that ie

                                                                                ddX

                                                                                d

                                                                                D

                                                                                0

                                                                                01

                                                                                ddB

                                                                                ddtt IBBBB

                                                                                DBBX tBDBX

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                (looks similar)

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                Complex (uses )

                                                                                VU

                                                                                1i

                                                                                Review of Linear Algebra (Cont)

                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                VU

                                                                                X

                                                                                BVU

                                                                                1i

                                                                                • Participant Presentations
                                                                                • PCA to find clusters
                                                                                • Statistical Smoothing
                                                                                • Kernel Density Estimation
                                                                                • Scatterplot Smoothing
                                                                                • Statistical Smoothing (2)
                                                                                • SiZer Background
                                                                                • SiZer Background (2)
                                                                                • SiZer Background (3)
                                                                                • SiZer Background (4)
                                                                                • SiZer Background (5)
                                                                                • SiZer Overview
                                                                                • PCA to find clusters (2)
                                                                                • PCA to find clusters (3)
                                                                                • PCA to find clusters (4)
                                                                                • PCA to find clusters (5)
                                                                                • PCA to find clusters (6)
                                                                                • PCA to find clusters (7)
                                                                                • Recall Yeast Cell Cycle Data
                                                                                • Yeast Cell Cycle Data FDA View
                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                • Frequency 2 Analysis
                                                                                • Frequency 2 Analysis (2)
                                                                                • Frequency 2 Analysis (3)
                                                                                • Yeast Cell Cycle
                                                                                • SiZer Study of Distrsquon of Angles
                                                                                • Reclassification of Major Genes
                                                                                • Compare to Previous Classifrsquon
                                                                                • New Subpopulation View
                                                                                • New Subpopulation View (2)
                                                                                • Detailed Look at PCA
                                                                                • PCA Rediscovery ndash Renaming
                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                • An Interesting Historical Note
                                                                                • An Interesting Historical Note (2)
                                                                                • Detailed Look at PCA (2)
                                                                                • Detailed Look at PCA (3)
                                                                                • Review of Linear Algebra
                                                                                • Review of Linear Algebra (2)
                                                                                • Review of Linear Algebra (3)
                                                                                • Review of Linear Algebra (Cont)
                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                  Detailed Look at PCA

                                                                                  Three Important (amp Interesting) Viewpoints

                                                                                  1 Mathematics

                                                                                  2 Numerics

                                                                                  3 Statistics

                                                                                  Goal Study Interrelationships

                                                                                  Detailed Look at PCA

                                                                                  Three Important (amp Interesting) Viewpoints

                                                                                  1 Mathematics

                                                                                  2 Numerics

                                                                                  3 Statistics

                                                                                  1st Review Linear Alg and Multivar Prob

                                                                                  Review of Linear Algebra

                                                                                  Vector Space

                                                                                  bull set of ldquovectorsrdquo

                                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                                  xa

                                                                                  Review of Linear Algebra

                                                                                  Vector Space

                                                                                  bull set of ldquovectorsrdquo

                                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                  ( in space)

                                                                                  xa

                                                                                  i

                                                                                  ii xa

                                                                                  Review of Linear Algebra

                                                                                  Vector Space

                                                                                  bull set of ldquovectorsrdquo

                                                                                  bull and ldquoscalarsrdquo (coefficients)

                                                                                  bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                  ( in space)

                                                                                  eg

                                                                                  ldquo dim Euclidrsquon spacerdquo

                                                                                  xa

                                                                                  i

                                                                                  ii xa

                                                                                  d

                                                                                  d

                                                                                  d xx

                                                                                  x

                                                                                  x

                                                                                  x 1

                                                                                  1

                                                                                  d

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                  Note Planes not Through the Origin

                                                                                  are not Subspaces

                                                                                  (Do not Contain )00 x

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                  (all Linear Combos of them =

                                                                                  = Containing Hyperplane

                                                                                  through Origin)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Basis of Subspace Set of Vectors that

                                                                                  bull Span ie Everything is a Lin Com of them

                                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Basis of Subspace Set of Vectors that

                                                                                  bull Span ie Everything is a Lin Com of them

                                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                  bull eg ldquoUnit Vector Basisrdquo

                                                                                  bull Since

                                                                                  d

                                                                                  1

                                                                                  0

                                                                                  0

                                                                                  0

                                                                                  1

                                                                                  0

                                                                                  0

                                                                                  0

                                                                                  1

                                                                                  1

                                                                                  0

                                                                                  0

                                                                                  0

                                                                                  1

                                                                                  0

                                                                                  0

                                                                                  0

                                                                                  1

                                                                                  212

                                                                                  1

                                                                                  d

                                                                                  d

                                                                                  xxx

                                                                                  x

                                                                                  x

                                                                                  x

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Basis Matrix of subspace of

                                                                                  Given a basis

                                                                                  create matrix of columns

                                                                                  dnvv 1

                                                                                  nddnd

                                                                                  n

                                                                                  n

                                                                                  vv

                                                                                  vv

                                                                                  vvB

                                                                                  1

                                                                                  111

                                                                                  1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                  where

                                                                                  n

                                                                                  iii aBva

                                                                                  1

                                                                                  na

                                                                                  a

                                                                                  a 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                  where

                                                                                  Note Right Multiplication Gives

                                                                                  Linear Combination of Column Vectors

                                                                                  n

                                                                                  iii aBva

                                                                                  1

                                                                                  na

                                                                                  a

                                                                                  a 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                  where

                                                                                  Check sizes

                                                                                  n

                                                                                  iii aBva

                                                                                  1

                                                                                  na

                                                                                  a

                                                                                  a 1

                                                                                  )1()(1 nndd

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                  For matrices

                                                                                  Define the Matrix Product

                                                                                  mkk

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  nmm

                                                                                  n

                                                                                  bb

                                                                                  bb

                                                                                  B

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iniik

                                                                                  m

                                                                                  iiik

                                                                                  m

                                                                                  inii

                                                                                  m

                                                                                  iii

                                                                                  baba

                                                                                  baba

                                                                                  AB

                                                                                  1

                                                                                  11

                                                                                  11

                                                                                  111

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                  For matrices

                                                                                  Define the Matrix Product

                                                                                  (Inner Products of Rows With Columns )

                                                                                  mkk

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  nmm

                                                                                  n

                                                                                  bb

                                                                                  bb

                                                                                  B

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iniik

                                                                                  m

                                                                                  iiik

                                                                                  m

                                                                                  inii

                                                                                  m

                                                                                  iii

                                                                                  baba

                                                                                  baba

                                                                                  AB

                                                                                  1

                                                                                  11

                                                                                  11

                                                                                  111

                                                                                  A B

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                  For matrices

                                                                                  Define the Matrix Product

                                                                                  (Inner Products of Rows With Columns )

                                                                                  (Composition of Linear Transformations)

                                                                                  mkk

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  nmm

                                                                                  n

                                                                                  bb

                                                                                  bb

                                                                                  B

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iniik

                                                                                  m

                                                                                  iiik

                                                                                  m

                                                                                  inii

                                                                                  m

                                                                                  iii

                                                                                  baba

                                                                                  baba

                                                                                  AB

                                                                                  1

                                                                                  11

                                                                                  11

                                                                                  111

                                                                                  A B

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                  For matrices

                                                                                  Define the Matrix Product

                                                                                  (Inner Products of Rows With Columns )

                                                                                  (Composition of Linear Transformations)

                                                                                  Often Useful to Check Sizes

                                                                                  mkk

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  nmm

                                                                                  n

                                                                                  bb

                                                                                  bb

                                                                                  B

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iniik

                                                                                  m

                                                                                  iiik

                                                                                  m

                                                                                  inii

                                                                                  m

                                                                                  iii

                                                                                  baba

                                                                                  baba

                                                                                  AB

                                                                                  1

                                                                                  11

                                                                                  11

                                                                                  111

                                                                                  nmmknk

                                                                                  A B

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                  For matrices

                                                                                  Define the Matrix Product

                                                                                  (Inner Products of Rows With Columns )

                                                                                  (Composition of Linear Transformations)

                                                                                  Often Useful to Check Sizes

                                                                                  mkk

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  nmm

                                                                                  n

                                                                                  bb

                                                                                  bb

                                                                                  B

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iniik

                                                                                  m

                                                                                  iiik

                                                                                  m

                                                                                  inii

                                                                                  m

                                                                                  iii

                                                                                  baba

                                                                                  baba

                                                                                  AB

                                                                                  1

                                                                                  11

                                                                                  11

                                                                                  111

                                                                                  nmmknk

                                                                                  A B

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Matrix Trace

                                                                                  bull For a Square Matrix

                                                                                  bull Define

                                                                                  mmm

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iiiaAtr

                                                                                  1)(

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Matrix Trace

                                                                                  bull For a Square Matrix

                                                                                  bull Define

                                                                                  bull Trace Commutes with Matrix Multiplication

                                                                                  mmm

                                                                                  m

                                                                                  aa

                                                                                  aa

                                                                                  A

                                                                                  1

                                                                                  111

                                                                                  m

                                                                                  iiiaAtr

                                                                                  1)(

                                                                                  BAtrABtr

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                  bull (Use Basis Above)

                                                                                  bull eg dim of a line is 1

                                                                                  bull eg dim of a plane is 2

                                                                                  dd dim

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                  bull (Use Basis Above)

                                                                                  bull eg dim of a line is 1

                                                                                  bull eg dim of a plane is 2

                                                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                  (in Statistical Uses eg ANOVA)

                                                                                  dd dim

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Norm of a Vector

                                                                                  bull in d 21

                                                                                  21

                                                                                  1

                                                                                  2 xxxx td

                                                                                  jj

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Norm of a Vector

                                                                                  bull in

                                                                                  bull Idea length of the vector

                                                                                  d 2121

                                                                                  1

                                                                                  2 xxxx td

                                                                                  jj

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Norm of a Vector

                                                                                  bull in

                                                                                  bull Idea length of the vector

                                                                                  bull Note strange properties for high

                                                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                                                  d 2121

                                                                                  1

                                                                                  2 xxxx td

                                                                                  jj

                                                                                  d

                                                                                  d

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Norm of a Vector (cont)

                                                                                  bull Length Normalized Vector

                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                  amp is a Direction Vector)

                                                                                  x

                                                                                  x

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Norm of a Vector (cont)

                                                                                  bull Length Normalized Vector

                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                  amp is a Direction Vector)

                                                                                  bull Define Distance as

                                                                                  x

                                                                                  x

                                                                                  yxyxyxyxd t

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Inner (Dot Scalar) Product

                                                                                  bull for Vectors and

                                                                                  yxyxyx td

                                                                                  jjj

                                                                                  1

                                                                                  x y

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Inner (Dot Scalar) Product

                                                                                  bull for Vectors and

                                                                                  bull Related to Norm via

                                                                                  yxyxyx td

                                                                                  jjj

                                                                                  1

                                                                                  x y

                                                                                  21 xxx

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                  bull measures ldquoangle between and rdquo as

                                                                                  yyxx

                                                                                  yx

                                                                                  yx

                                                                                  yxyxangle

                                                                                  tt

                                                                                  t

                                                                                  11 cos

                                                                                  cos

                                                                                  x y

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                  bull measures ldquoangle between and rdquo as

                                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                                  if and only if

                                                                                  yyxx

                                                                                  yx

                                                                                  yx

                                                                                  yxyxangle

                                                                                  tt

                                                                                  t

                                                                                  11 cos

                                                                                  cos

                                                                                  x y

                                                                                  yx 0 yx

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis

                                                                                  bull All Orthogonal to each other

                                                                                  ie for

                                                                                  bull All have Length 1

                                                                                  ie for

                                                                                  nvv 1

                                                                                  1 ii vv

                                                                                  0 ii vv ii

                                                                                  ni 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis (cont)

                                                                                  bull Spectral Representation

                                                                                  where

                                                                                  nvv 1

                                                                                  n

                                                                                  iii vax

                                                                                  1

                                                                                  ii vxa

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis (cont)

                                                                                  bull Spectral Representation

                                                                                  where

                                                                                  (Coefficient is Inner Product Cool Notation)

                                                                                  nvv 1

                                                                                  n

                                                                                  iii vax

                                                                                  1

                                                                                  ii vxa

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis (cont)

                                                                                  bull Spectral Representation

                                                                                  where

                                                                                  Check

                                                                                  nvv 1

                                                                                  n

                                                                                  iii vax

                                                                                  1

                                                                                  ii vxa

                                                                                  iii

                                                                                  n

                                                                                  iii

                                                                                  n

                                                                                  iiii avvavvavx

                                                                                  1

                                                                                  1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis (cont)

                                                                                  bull Spectral Representation

                                                                                  where

                                                                                  Check

                                                                                  bull Matrix Notation where ie

                                                                                  For the Basis Matrix

                                                                                  nvv 1

                                                                                  n

                                                                                  iii vax

                                                                                  1

                                                                                  ii vxa

                                                                                  iii

                                                                                  n

                                                                                  iii

                                                                                  n

                                                                                  iiii avvavvavx

                                                                                  1

                                                                                  1

                                                                                  aBx Bxa tt xBa t

                                                                                  nvvB 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Orthonormal Basis (cont)

                                                                                  bull Spectral Representation

                                                                                  where

                                                                                  Check

                                                                                  bull Matrix Notation where ie

                                                                                  is called transform of

                                                                                  (eg Fourier or Wavelet)

                                                                                  nvv 1

                                                                                  n

                                                                                  iii vax

                                                                                  1

                                                                                  ii vxa

                                                                                  iii

                                                                                  n

                                                                                  iii

                                                                                  n

                                                                                  iiii avvavvavx

                                                                                  1

                                                                                  1

                                                                                  aBx Bxa tt xBa t

                                                                                  xa

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Parseval identity for

                                                                                  in subsp genrsquod by o n basis

                                                                                  x

                                                                                  nvv 1

                                                                                  2

                                                                                  1

                                                                                  22

                                                                                  1

                                                                                  2 aavxx

                                                                                  n

                                                                                  ii

                                                                                  n

                                                                                  ii

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Parseval identity for

                                                                                  in subsp genrsquod by o n basis

                                                                                  bull Pythagorean theorem

                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                  bull ANOVA - sums of squares

                                                                                  x

                                                                                  nvv 1

                                                                                  2

                                                                                  1

                                                                                  22

                                                                                  1

                                                                                  2 aavxx

                                                                                  n

                                                                                  ii

                                                                                  n

                                                                                  ii

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Parseval identity for

                                                                                  in subsp genrsquod by o n basis

                                                                                  bull Pythagorean theorem

                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                  bull ANOVA - sums of squares

                                                                                  bull Transform has same length as

                                                                                  ie ldquorotation in rdquo

                                                                                  x

                                                                                  nvv 1

                                                                                  2

                                                                                  1

                                                                                  22

                                                                                  1

                                                                                  2 aavxx

                                                                                  n

                                                                                  ii

                                                                                  n

                                                                                  ii

                                                                                  a xd

                                                                                  Projection of a Vector onto a Subspace

                                                                                  bull Idea Member of that is Closest to

                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                  Review of Linear Algebra (Cont)x

                                                                                  xV

                                                                                  V

                                                                                  Projection of a Vector onto a Subspace

                                                                                  bull Idea Member of that is Closest to

                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                  bull Find that Solves

                                                                                  (ldquoLeast Squaresrdquo)

                                                                                  Review of Linear Algebra (Cont)x

                                                                                  xV

                                                                                  V

                                                                                  VxPV vxVv

                                                                                  min

                                                                                  Projection of a Vector onto a Subspace

                                                                                  bull Idea Member of that is Closest to

                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                  bull Find that Solves

                                                                                  (ldquoLeast Squaresrdquo)

                                                                                  bull For Inner Product (Hilbert) Space

                                                                                  Exists and is Unique

                                                                                  Review of Linear Algebra (Cont)x

                                                                                  xV

                                                                                  V

                                                                                  VxPV vxVv

                                                                                  min

                                                                                  xPV

                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                  bull General Solution in for Basis Matrix

                                                                                  Review of Linear Algebra (Cont)

                                                                                  d VB

                                                                                  xBBBBxP tVV

                                                                                  tVVV

                                                                                  1

                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                  bull General Solution in for Basis Matrix

                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                  (thus projection is another linear operation)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  d VB

                                                                                  xBBBBxP tVV

                                                                                  tVVV

                                                                                  1

                                                                                  tVV

                                                                                  tVVV BBBBP

                                                                                  1

                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                  bull General Solution in for Basis Matrix

                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                  (thus projection is another linear operation)

                                                                                  (note same operation underlies least squares)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  d VB

                                                                                  xBBBBxP tVV

                                                                                  tVVV

                                                                                  1

                                                                                  tVV

                                                                                  tVVV BBBBP

                                                                                  1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Projection using Orthonormal Basis

                                                                                  bull Basis Matrix is Orthonormal

                                                                                  nnVtV IBB

                                                                                  10

                                                                                  01

                                                                                  1

                                                                                  111

                                                                                  1

                                                                                  1

                                                                                  nnn

                                                                                  n

                                                                                  ntn

                                                                                  t

                                                                                  vvvv

                                                                                  vvvv

                                                                                  vv

                                                                                  v

                                                                                  v

                                                                                  nvv 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Projection using Orthonormal Basis

                                                                                  bull Basis Matrix is Orthonormal

                                                                                  bull So =

                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                  (Recall Right Multrsquon)

                                                                                  nnVtV IBB

                                                                                  10

                                                                                  01

                                                                                  1

                                                                                  111

                                                                                  1

                                                                                  1

                                                                                  nnn

                                                                                  n

                                                                                  ntn

                                                                                  t

                                                                                  vvvv

                                                                                  vvvv

                                                                                  vv

                                                                                  v

                                                                                  v

                                                                                  xBBxP tVVV

                                                                                  x V

                                                                                  nvv 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Projection using Orthonormal Basis (cont)

                                                                                  bull For Orthogonal Complement

                                                                                  and

                                                                                  V

                                                                                  xPxPx VV 222xPxPx VV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Projection using Orthonormal Basis (cont)

                                                                                  bull For Orthogonal Complement

                                                                                  and

                                                                                  bull Parseval Inequality

                                                                                  V

                                                                                  xPxPx VV 222xPxPx VV

                                                                                  2

                                                                                  1

                                                                                  22

                                                                                  1

                                                                                  22 aavxxxP

                                                                                  n

                                                                                  ii

                                                                                  n

                                                                                  iiV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  (Real) Unitary Matrices with

                                                                                  bull Orthonormal Basis Matrix

                                                                                  (So All of Above Applies)

                                                                                  ddU IUU t

                                                                                  Review of Linear Algebra (Cont)

                                                                                  (Real) Unitary Matrices with

                                                                                  bull Orthonormal Basis Matrix

                                                                                  (So All of Above Applies)

                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                  ddU IUU t

                                                                                  yxdyxyxyxUyUxUdn

                                                                                  i ii 2

                                                                                  1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  (Real) Unitary Matrices with

                                                                                  bull Orthonormal Basis Matrix

                                                                                  (So All of Above Applies)

                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                  ddU IUU t

                                                                                  yxdyxyxyxUyUxUdn

                                                                                  i ii 2

                                                                                  1

                                                                                  U

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Singular Value Decomposition (SVD)

                                                                                  For a Matrix

                                                                                  Find

                                                                                  ndX

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Singular Value Decomposition (SVD)

                                                                                  For a Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  with Entries

                                                                                  called Singular Values

                                                                                  ndX

                                                                                  ndS

                                                                                  )min(1 ndss

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Singular Value Decomposition (SVD)

                                                                                  For a Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  with Entries

                                                                                  called Singular Values

                                                                                  And Unitary (Rotation) Matrices

                                                                                  (recall )

                                                                                  ndX

                                                                                  ndS

                                                                                  )min(1 ndss

                                                                                  ddU nnV

                                                                                  IVVUU tt

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Singular Value Decomposition (SVD)

                                                                                  For a Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  with Entries

                                                                                  called Singular Values

                                                                                  And Unitary (Rotation) Matrices

                                                                                  (recall )

                                                                                  So That

                                                                                  ndX

                                                                                  ndS

                                                                                  )min(1 ndss

                                                                                  ddU nnV

                                                                                  IVVUU tt tUSVX

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Intuition behind Singular Value Decomposition

                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                  vVSUvVSUvX tt

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Intuition behind Singular Value Decomposition

                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                  bull First rotate

                                                                                  X

                                                                                  vVSUvVSUvX tt

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Intuition behind Singular Value Decomposition

                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                  bull First rotate

                                                                                  bull Second rescale coordinate axes (by )

                                                                                  X

                                                                                  vVSUvVSUvX tt

                                                                                  is

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Intuition behind Singular Value Decomposition

                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                  bull First rotate

                                                                                  bull Second rescale coordinate axes (by )

                                                                                  bull Third rotate again

                                                                                  X

                                                                                  vVSUvVSUvX tt

                                                                                  is

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Intuition behind Singular Value Decomposition

                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                  bull First rotate

                                                                                  bull Second rescale coordinate axes (by )

                                                                                  bull Third rotate again

                                                                                  bull ie have diagonalized the transformation

                                                                                  X

                                                                                  vVSUvVSUvX tt

                                                                                  is

                                                                                  SVD Compact Representation

                                                                                  Useful Labeling

                                                                                  Singular Values in Increasing Order

                                                                                  Review of Linear Algebra (Cont)

                                                                                  )min(1 dnss

                                                                                  SVD Compact Representation

                                                                                  Useful Labeling

                                                                                  Singular Values in Increasing Order

                                                                                  Note singular values = 0 can be omitted

                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  )min(1 dnss

                                                                                  Review of Linear Algebra (Cont)

                                                                                  r

                                                                                  SVD Compact Representation

                                                                                  Useful Labeling

                                                                                  Singular Values in Increasing Order

                                                                                  Note singular values = 0 can be omitted

                                                                                  Let = of positive singular values

                                                                                  )min(1 dnss

                                                                                  Review of Linear Algebra (Cont)

                                                                                  r

                                                                                  SVD Compact Representation

                                                                                  Useful Labeling

                                                                                  Singular Values in Increasing Order

                                                                                  Note singular values = 0 can be omitted

                                                                                  Let = of positive singular values

                                                                                  Then

                                                                                  Where are truncations of

                                                                                  trnrrrd VSUX

                                                                                  VSU

                                                                                  )min(1 dnss

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Full Representation

                                                                                  =

                                                                                  Graphics Display Assumes

                                                                                  ndX ddU ndS nn

                                                                                  tV

                                                                                  nd

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Full Representation

                                                                                  =

                                                                                  Full Rank Basis Matrix

                                                                                  ndX ddU ndS nn

                                                                                  tV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Full Representation

                                                                                  =

                                                                                  Full Rank Basis Matrix

                                                                                  All 0s in Bottom

                                                                                  ndX ddU ndS nn

                                                                                  tV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Reduced Representation

                                                                                  =

                                                                                  These Columns Get 0ed Out

                                                                                  ndX ddU nnS nn

                                                                                  tV

                                                                                  nnd 0

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Reduced Representation

                                                                                  =ndX ndU nnS nn

                                                                                  tV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Reduced Representation

                                                                                  =

                                                                                  Also Some of These May be 0

                                                                                  ndX ndU nnS nn

                                                                                  tV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Compact Representation

                                                                                  = ndX rdU

                                                                                  rrS nrtV

                                                                                  0

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Compact Representation

                                                                                  =

                                                                                  These Get 0ed Out

                                                                                  ndX rdU

                                                                                  rrS nrtV

                                                                                  0

                                                                                  Review of Linear Algebra (Cont)

                                                                                  SVD Compact Representation

                                                                                  = ndX rdU

                                                                                  rrS nrtV

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition

                                                                                  For a (Symmetric) Square Matrix

                                                                                  Find

                                                                                  ddX

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition

                                                                                  For a (Symmetric) Square Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  Called Eigenvalues

                                                                                  Convenient Ordering

                                                                                  ddX

                                                                                  d

                                                                                  D

                                                                                  0

                                                                                  01

                                                                                  n 1

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition

                                                                                  For a (Symmetric) Square Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  And an Orthonormal Matrix

                                                                                  (ie )

                                                                                  ddX

                                                                                  d

                                                                                  D

                                                                                  0

                                                                                  01

                                                                                  ddB

                                                                                  ddtt IBBBB

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition

                                                                                  For a (Symmetric) Square Matrix

                                                                                  Find a Diagonal Matrix

                                                                                  And an Orthonormal Matrix

                                                                                  (ie )

                                                                                  So that ie

                                                                                  ddX

                                                                                  d

                                                                                  D

                                                                                  0

                                                                                  01

                                                                                  ddB

                                                                                  ddtt IBBBB

                                                                                  DBBX tBDBX

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                  (looks similar)

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                  Complex (uses )

                                                                                  VU

                                                                                  1i

                                                                                  Review of Linear Algebra (Cont)

                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                  VU

                                                                                  X

                                                                                  BVU

                                                                                  1i

                                                                                  • Participant Presentations
                                                                                  • PCA to find clusters
                                                                                  • Statistical Smoothing
                                                                                  • Kernel Density Estimation
                                                                                  • Scatterplot Smoothing
                                                                                  • Statistical Smoothing (2)
                                                                                  • SiZer Background
                                                                                  • SiZer Background (2)
                                                                                  • SiZer Background (3)
                                                                                  • SiZer Background (4)
                                                                                  • SiZer Background (5)
                                                                                  • SiZer Overview
                                                                                  • PCA to find clusters (2)
                                                                                  • PCA to find clusters (3)
                                                                                  • PCA to find clusters (4)
                                                                                  • PCA to find clusters (5)
                                                                                  • PCA to find clusters (6)
                                                                                  • PCA to find clusters (7)
                                                                                  • Recall Yeast Cell Cycle Data
                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                  • Frequency 2 Analysis
                                                                                  • Frequency 2 Analysis (2)
                                                                                  • Frequency 2 Analysis (3)
                                                                                  • Yeast Cell Cycle
                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                  • Reclassification of Major Genes
                                                                                  • Compare to Previous Classifrsquon
                                                                                  • New Subpopulation View
                                                                                  • New Subpopulation View (2)
                                                                                  • Detailed Look at PCA
                                                                                  • PCA Rediscovery ndash Renaming
                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                  • An Interesting Historical Note
                                                                                  • An Interesting Historical Note (2)
                                                                                  • Detailed Look at PCA (2)
                                                                                  • Detailed Look at PCA (3)
                                                                                  • Review of Linear Algebra
                                                                                  • Review of Linear Algebra (2)
                                                                                  • Review of Linear Algebra (3)
                                                                                  • Review of Linear Algebra (Cont)
                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                    Detailed Look at PCA

                                                                                    Three Important (amp Interesting) Viewpoints

                                                                                    1 Mathematics

                                                                                    2 Numerics

                                                                                    3 Statistics

                                                                                    1st Review Linear Alg and Multivar Prob

                                                                                    Review of Linear Algebra

                                                                                    Vector Space

                                                                                    bull set of ldquovectorsrdquo

                                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                                    xa

                                                                                    Review of Linear Algebra

                                                                                    Vector Space

                                                                                    bull set of ldquovectorsrdquo

                                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                    ( in space)

                                                                                    xa

                                                                                    i

                                                                                    ii xa

                                                                                    Review of Linear Algebra

                                                                                    Vector Space

                                                                                    bull set of ldquovectorsrdquo

                                                                                    bull and ldquoscalarsrdquo (coefficients)

                                                                                    bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                    ( in space)

                                                                                    eg

                                                                                    ldquo dim Euclidrsquon spacerdquo

                                                                                    xa

                                                                                    i

                                                                                    ii xa

                                                                                    d

                                                                                    d

                                                                                    d xx

                                                                                    x

                                                                                    x

                                                                                    x 1

                                                                                    1

                                                                                    d

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                    Note Planes not Through the Origin

                                                                                    are not Subspaces

                                                                                    (Do not Contain )00 x

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                    (all Linear Combos of them =

                                                                                    = Containing Hyperplane

                                                                                    through Origin)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Basis of Subspace Set of Vectors that

                                                                                    bull Span ie Everything is a Lin Com of them

                                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Basis of Subspace Set of Vectors that

                                                                                    bull Span ie Everything is a Lin Com of them

                                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                    bull eg ldquoUnit Vector Basisrdquo

                                                                                    bull Since

                                                                                    d

                                                                                    1

                                                                                    0

                                                                                    0

                                                                                    0

                                                                                    1

                                                                                    0

                                                                                    0

                                                                                    0

                                                                                    1

                                                                                    1

                                                                                    0

                                                                                    0

                                                                                    0

                                                                                    1

                                                                                    0

                                                                                    0

                                                                                    0

                                                                                    1

                                                                                    212

                                                                                    1

                                                                                    d

                                                                                    d

                                                                                    xxx

                                                                                    x

                                                                                    x

                                                                                    x

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Basis Matrix of subspace of

                                                                                    Given a basis

                                                                                    create matrix of columns

                                                                                    dnvv 1

                                                                                    nddnd

                                                                                    n

                                                                                    n

                                                                                    vv

                                                                                    vv

                                                                                    vvB

                                                                                    1

                                                                                    111

                                                                                    1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                    where

                                                                                    n

                                                                                    iii aBva

                                                                                    1

                                                                                    na

                                                                                    a

                                                                                    a 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                    where

                                                                                    Note Right Multiplication Gives

                                                                                    Linear Combination of Column Vectors

                                                                                    n

                                                                                    iii aBva

                                                                                    1

                                                                                    na

                                                                                    a

                                                                                    a 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                    where

                                                                                    Check sizes

                                                                                    n

                                                                                    iii aBva

                                                                                    1

                                                                                    na

                                                                                    a

                                                                                    a 1

                                                                                    )1()(1 nndd

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                    For matrices

                                                                                    Define the Matrix Product

                                                                                    mkk

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    nmm

                                                                                    n

                                                                                    bb

                                                                                    bb

                                                                                    B

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iniik

                                                                                    m

                                                                                    iiik

                                                                                    m

                                                                                    inii

                                                                                    m

                                                                                    iii

                                                                                    baba

                                                                                    baba

                                                                                    AB

                                                                                    1

                                                                                    11

                                                                                    11

                                                                                    111

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                    For matrices

                                                                                    Define the Matrix Product

                                                                                    (Inner Products of Rows With Columns )

                                                                                    mkk

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    nmm

                                                                                    n

                                                                                    bb

                                                                                    bb

                                                                                    B

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iniik

                                                                                    m

                                                                                    iiik

                                                                                    m

                                                                                    inii

                                                                                    m

                                                                                    iii

                                                                                    baba

                                                                                    baba

                                                                                    AB

                                                                                    1

                                                                                    11

                                                                                    11

                                                                                    111

                                                                                    A B

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                    For matrices

                                                                                    Define the Matrix Product

                                                                                    (Inner Products of Rows With Columns )

                                                                                    (Composition of Linear Transformations)

                                                                                    mkk

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    nmm

                                                                                    n

                                                                                    bb

                                                                                    bb

                                                                                    B

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iniik

                                                                                    m

                                                                                    iiik

                                                                                    m

                                                                                    inii

                                                                                    m

                                                                                    iii

                                                                                    baba

                                                                                    baba

                                                                                    AB

                                                                                    1

                                                                                    11

                                                                                    11

                                                                                    111

                                                                                    A B

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                    For matrices

                                                                                    Define the Matrix Product

                                                                                    (Inner Products of Rows With Columns )

                                                                                    (Composition of Linear Transformations)

                                                                                    Often Useful to Check Sizes

                                                                                    mkk

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    nmm

                                                                                    n

                                                                                    bb

                                                                                    bb

                                                                                    B

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iniik

                                                                                    m

                                                                                    iiik

                                                                                    m

                                                                                    inii

                                                                                    m

                                                                                    iii

                                                                                    baba

                                                                                    baba

                                                                                    AB

                                                                                    1

                                                                                    11

                                                                                    11

                                                                                    111

                                                                                    nmmknk

                                                                                    A B

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                    For matrices

                                                                                    Define the Matrix Product

                                                                                    (Inner Products of Rows With Columns )

                                                                                    (Composition of Linear Transformations)

                                                                                    Often Useful to Check Sizes

                                                                                    mkk

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    nmm

                                                                                    n

                                                                                    bb

                                                                                    bb

                                                                                    B

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iniik

                                                                                    m

                                                                                    iiik

                                                                                    m

                                                                                    inii

                                                                                    m

                                                                                    iii

                                                                                    baba

                                                                                    baba

                                                                                    AB

                                                                                    1

                                                                                    11

                                                                                    11

                                                                                    111

                                                                                    nmmknk

                                                                                    A B

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Matrix Trace

                                                                                    bull For a Square Matrix

                                                                                    bull Define

                                                                                    mmm

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iiiaAtr

                                                                                    1)(

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Matrix Trace

                                                                                    bull For a Square Matrix

                                                                                    bull Define

                                                                                    bull Trace Commutes with Matrix Multiplication

                                                                                    mmm

                                                                                    m

                                                                                    aa

                                                                                    aa

                                                                                    A

                                                                                    1

                                                                                    111

                                                                                    m

                                                                                    iiiaAtr

                                                                                    1)(

                                                                                    BAtrABtr

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                    bull (Use Basis Above)

                                                                                    bull eg dim of a line is 1

                                                                                    bull eg dim of a plane is 2

                                                                                    dd dim

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                    bull (Use Basis Above)

                                                                                    bull eg dim of a line is 1

                                                                                    bull eg dim of a plane is 2

                                                                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                    (in Statistical Uses eg ANOVA)

                                                                                    dd dim

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Norm of a Vector

                                                                                    bull in d 21

                                                                                    21

                                                                                    1

                                                                                    2 xxxx td

                                                                                    jj

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Norm of a Vector

                                                                                    bull in

                                                                                    bull Idea length of the vector

                                                                                    d 2121

                                                                                    1

                                                                                    2 xxxx td

                                                                                    jj

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Norm of a Vector

                                                                                    bull in

                                                                                    bull Idea length of the vector

                                                                                    bull Note strange properties for high

                                                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                                                    d 2121

                                                                                    1

                                                                                    2 xxxx td

                                                                                    jj

                                                                                    d

                                                                                    d

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Norm of a Vector (cont)

                                                                                    bull Length Normalized Vector

                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                    amp is a Direction Vector)

                                                                                    x

                                                                                    x

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Norm of a Vector (cont)

                                                                                    bull Length Normalized Vector

                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                    amp is a Direction Vector)

                                                                                    bull Define Distance as

                                                                                    x

                                                                                    x

                                                                                    yxyxyxyxd t

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Inner (Dot Scalar) Product

                                                                                    bull for Vectors and

                                                                                    yxyxyx td

                                                                                    jjj

                                                                                    1

                                                                                    x y

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Inner (Dot Scalar) Product

                                                                                    bull for Vectors and

                                                                                    bull Related to Norm via

                                                                                    yxyxyx td

                                                                                    jjj

                                                                                    1

                                                                                    x y

                                                                                    21 xxx

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                    bull measures ldquoangle between and rdquo as

                                                                                    yyxx

                                                                                    yx

                                                                                    yx

                                                                                    yxyxangle

                                                                                    tt

                                                                                    t

                                                                                    11 cos

                                                                                    cos

                                                                                    x y

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                    bull measures ldquoangle between and rdquo as

                                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                                    if and only if

                                                                                    yyxx

                                                                                    yx

                                                                                    yx

                                                                                    yxyxangle

                                                                                    tt

                                                                                    t

                                                                                    11 cos

                                                                                    cos

                                                                                    x y

                                                                                    yx 0 yx

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis

                                                                                    bull All Orthogonal to each other

                                                                                    ie for

                                                                                    bull All have Length 1

                                                                                    ie for

                                                                                    nvv 1

                                                                                    1 ii vv

                                                                                    0 ii vv ii

                                                                                    ni 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis (cont)

                                                                                    bull Spectral Representation

                                                                                    where

                                                                                    nvv 1

                                                                                    n

                                                                                    iii vax

                                                                                    1

                                                                                    ii vxa

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis (cont)

                                                                                    bull Spectral Representation

                                                                                    where

                                                                                    (Coefficient is Inner Product Cool Notation)

                                                                                    nvv 1

                                                                                    n

                                                                                    iii vax

                                                                                    1

                                                                                    ii vxa

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis (cont)

                                                                                    bull Spectral Representation

                                                                                    where

                                                                                    Check

                                                                                    nvv 1

                                                                                    n

                                                                                    iii vax

                                                                                    1

                                                                                    ii vxa

                                                                                    iii

                                                                                    n

                                                                                    iii

                                                                                    n

                                                                                    iiii avvavvavx

                                                                                    1

                                                                                    1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis (cont)

                                                                                    bull Spectral Representation

                                                                                    where

                                                                                    Check

                                                                                    bull Matrix Notation where ie

                                                                                    For the Basis Matrix

                                                                                    nvv 1

                                                                                    n

                                                                                    iii vax

                                                                                    1

                                                                                    ii vxa

                                                                                    iii

                                                                                    n

                                                                                    iii

                                                                                    n

                                                                                    iiii avvavvavx

                                                                                    1

                                                                                    1

                                                                                    aBx Bxa tt xBa t

                                                                                    nvvB 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Orthonormal Basis (cont)

                                                                                    bull Spectral Representation

                                                                                    where

                                                                                    Check

                                                                                    bull Matrix Notation where ie

                                                                                    is called transform of

                                                                                    (eg Fourier or Wavelet)

                                                                                    nvv 1

                                                                                    n

                                                                                    iii vax

                                                                                    1

                                                                                    ii vxa

                                                                                    iii

                                                                                    n

                                                                                    iii

                                                                                    n

                                                                                    iiii avvavvavx

                                                                                    1

                                                                                    1

                                                                                    aBx Bxa tt xBa t

                                                                                    xa

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Parseval identity for

                                                                                    in subsp genrsquod by o n basis

                                                                                    x

                                                                                    nvv 1

                                                                                    2

                                                                                    1

                                                                                    22

                                                                                    1

                                                                                    2 aavxx

                                                                                    n

                                                                                    ii

                                                                                    n

                                                                                    ii

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Parseval identity for

                                                                                    in subsp genrsquod by o n basis

                                                                                    bull Pythagorean theorem

                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                    bull ANOVA - sums of squares

                                                                                    x

                                                                                    nvv 1

                                                                                    2

                                                                                    1

                                                                                    22

                                                                                    1

                                                                                    2 aavxx

                                                                                    n

                                                                                    ii

                                                                                    n

                                                                                    ii

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Parseval identity for

                                                                                    in subsp genrsquod by o n basis

                                                                                    bull Pythagorean theorem

                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                    bull ANOVA - sums of squares

                                                                                    bull Transform has same length as

                                                                                    ie ldquorotation in rdquo

                                                                                    x

                                                                                    nvv 1

                                                                                    2

                                                                                    1

                                                                                    22

                                                                                    1

                                                                                    2 aavxx

                                                                                    n

                                                                                    ii

                                                                                    n

                                                                                    ii

                                                                                    a xd

                                                                                    Projection of a Vector onto a Subspace

                                                                                    bull Idea Member of that is Closest to

                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                    Review of Linear Algebra (Cont)x

                                                                                    xV

                                                                                    V

                                                                                    Projection of a Vector onto a Subspace

                                                                                    bull Idea Member of that is Closest to

                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                    bull Find that Solves

                                                                                    (ldquoLeast Squaresrdquo)

                                                                                    Review of Linear Algebra (Cont)x

                                                                                    xV

                                                                                    V

                                                                                    VxPV vxVv

                                                                                    min

                                                                                    Projection of a Vector onto a Subspace

                                                                                    bull Idea Member of that is Closest to

                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                    bull Find that Solves

                                                                                    (ldquoLeast Squaresrdquo)

                                                                                    bull For Inner Product (Hilbert) Space

                                                                                    Exists and is Unique

                                                                                    Review of Linear Algebra (Cont)x

                                                                                    xV

                                                                                    V

                                                                                    VxPV vxVv

                                                                                    min

                                                                                    xPV

                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                    bull General Solution in for Basis Matrix

                                                                                    Review of Linear Algebra (Cont)

                                                                                    d VB

                                                                                    xBBBBxP tVV

                                                                                    tVVV

                                                                                    1

                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                    bull General Solution in for Basis Matrix

                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                    (thus projection is another linear operation)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    d VB

                                                                                    xBBBBxP tVV

                                                                                    tVVV

                                                                                    1

                                                                                    tVV

                                                                                    tVVV BBBBP

                                                                                    1

                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                    bull General Solution in for Basis Matrix

                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                    (thus projection is another linear operation)

                                                                                    (note same operation underlies least squares)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    d VB

                                                                                    xBBBBxP tVV

                                                                                    tVVV

                                                                                    1

                                                                                    tVV

                                                                                    tVVV BBBBP

                                                                                    1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Projection using Orthonormal Basis

                                                                                    bull Basis Matrix is Orthonormal

                                                                                    nnVtV IBB

                                                                                    10

                                                                                    01

                                                                                    1

                                                                                    111

                                                                                    1

                                                                                    1

                                                                                    nnn

                                                                                    n

                                                                                    ntn

                                                                                    t

                                                                                    vvvv

                                                                                    vvvv

                                                                                    vv

                                                                                    v

                                                                                    v

                                                                                    nvv 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Projection using Orthonormal Basis

                                                                                    bull Basis Matrix is Orthonormal

                                                                                    bull So =

                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                    (Recall Right Multrsquon)

                                                                                    nnVtV IBB

                                                                                    10

                                                                                    01

                                                                                    1

                                                                                    111

                                                                                    1

                                                                                    1

                                                                                    nnn

                                                                                    n

                                                                                    ntn

                                                                                    t

                                                                                    vvvv

                                                                                    vvvv

                                                                                    vv

                                                                                    v

                                                                                    v

                                                                                    xBBxP tVVV

                                                                                    x V

                                                                                    nvv 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Projection using Orthonormal Basis (cont)

                                                                                    bull For Orthogonal Complement

                                                                                    and

                                                                                    V

                                                                                    xPxPx VV 222xPxPx VV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Projection using Orthonormal Basis (cont)

                                                                                    bull For Orthogonal Complement

                                                                                    and

                                                                                    bull Parseval Inequality

                                                                                    V

                                                                                    xPxPx VV 222xPxPx VV

                                                                                    2

                                                                                    1

                                                                                    22

                                                                                    1

                                                                                    22 aavxxxP

                                                                                    n

                                                                                    ii

                                                                                    n

                                                                                    iiV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    (Real) Unitary Matrices with

                                                                                    bull Orthonormal Basis Matrix

                                                                                    (So All of Above Applies)

                                                                                    ddU IUU t

                                                                                    Review of Linear Algebra (Cont)

                                                                                    (Real) Unitary Matrices with

                                                                                    bull Orthonormal Basis Matrix

                                                                                    (So All of Above Applies)

                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                    ddU IUU t

                                                                                    yxdyxyxyxUyUxUdn

                                                                                    i ii 2

                                                                                    1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    (Real) Unitary Matrices with

                                                                                    bull Orthonormal Basis Matrix

                                                                                    (So All of Above Applies)

                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                    ddU IUU t

                                                                                    yxdyxyxyxUyUxUdn

                                                                                    i ii 2

                                                                                    1

                                                                                    U

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Singular Value Decomposition (SVD)

                                                                                    For a Matrix

                                                                                    Find

                                                                                    ndX

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Singular Value Decomposition (SVD)

                                                                                    For a Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    with Entries

                                                                                    called Singular Values

                                                                                    ndX

                                                                                    ndS

                                                                                    )min(1 ndss

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Singular Value Decomposition (SVD)

                                                                                    For a Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    with Entries

                                                                                    called Singular Values

                                                                                    And Unitary (Rotation) Matrices

                                                                                    (recall )

                                                                                    ndX

                                                                                    ndS

                                                                                    )min(1 ndss

                                                                                    ddU nnV

                                                                                    IVVUU tt

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Singular Value Decomposition (SVD)

                                                                                    For a Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    with Entries

                                                                                    called Singular Values

                                                                                    And Unitary (Rotation) Matrices

                                                                                    (recall )

                                                                                    So That

                                                                                    ndX

                                                                                    ndS

                                                                                    )min(1 ndss

                                                                                    ddU nnV

                                                                                    IVVUU tt tUSVX

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Intuition behind Singular Value Decomposition

                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                    vVSUvVSUvX tt

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Intuition behind Singular Value Decomposition

                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                    bull First rotate

                                                                                    X

                                                                                    vVSUvVSUvX tt

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Intuition behind Singular Value Decomposition

                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                    bull First rotate

                                                                                    bull Second rescale coordinate axes (by )

                                                                                    X

                                                                                    vVSUvVSUvX tt

                                                                                    is

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Intuition behind Singular Value Decomposition

                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                    bull First rotate

                                                                                    bull Second rescale coordinate axes (by )

                                                                                    bull Third rotate again

                                                                                    X

                                                                                    vVSUvVSUvX tt

                                                                                    is

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Intuition behind Singular Value Decomposition

                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                    bull First rotate

                                                                                    bull Second rescale coordinate axes (by )

                                                                                    bull Third rotate again

                                                                                    bull ie have diagonalized the transformation

                                                                                    X

                                                                                    vVSUvVSUvX tt

                                                                                    is

                                                                                    SVD Compact Representation

                                                                                    Useful Labeling

                                                                                    Singular Values in Increasing Order

                                                                                    Review of Linear Algebra (Cont)

                                                                                    )min(1 dnss

                                                                                    SVD Compact Representation

                                                                                    Useful Labeling

                                                                                    Singular Values in Increasing Order

                                                                                    Note singular values = 0 can be omitted

                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    )min(1 dnss

                                                                                    Review of Linear Algebra (Cont)

                                                                                    r

                                                                                    SVD Compact Representation

                                                                                    Useful Labeling

                                                                                    Singular Values in Increasing Order

                                                                                    Note singular values = 0 can be omitted

                                                                                    Let = of positive singular values

                                                                                    )min(1 dnss

                                                                                    Review of Linear Algebra (Cont)

                                                                                    r

                                                                                    SVD Compact Representation

                                                                                    Useful Labeling

                                                                                    Singular Values in Increasing Order

                                                                                    Note singular values = 0 can be omitted

                                                                                    Let = of positive singular values

                                                                                    Then

                                                                                    Where are truncations of

                                                                                    trnrrrd VSUX

                                                                                    VSU

                                                                                    )min(1 dnss

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Full Representation

                                                                                    =

                                                                                    Graphics Display Assumes

                                                                                    ndX ddU ndS nn

                                                                                    tV

                                                                                    nd

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Full Representation

                                                                                    =

                                                                                    Full Rank Basis Matrix

                                                                                    ndX ddU ndS nn

                                                                                    tV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Full Representation

                                                                                    =

                                                                                    Full Rank Basis Matrix

                                                                                    All 0s in Bottom

                                                                                    ndX ddU ndS nn

                                                                                    tV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Reduced Representation

                                                                                    =

                                                                                    These Columns Get 0ed Out

                                                                                    ndX ddU nnS nn

                                                                                    tV

                                                                                    nnd 0

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Reduced Representation

                                                                                    =ndX ndU nnS nn

                                                                                    tV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Reduced Representation

                                                                                    =

                                                                                    Also Some of These May be 0

                                                                                    ndX ndU nnS nn

                                                                                    tV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Compact Representation

                                                                                    = ndX rdU

                                                                                    rrS nrtV

                                                                                    0

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Compact Representation

                                                                                    =

                                                                                    These Get 0ed Out

                                                                                    ndX rdU

                                                                                    rrS nrtV

                                                                                    0

                                                                                    Review of Linear Algebra (Cont)

                                                                                    SVD Compact Representation

                                                                                    = ndX rdU

                                                                                    rrS nrtV

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition

                                                                                    For a (Symmetric) Square Matrix

                                                                                    Find

                                                                                    ddX

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition

                                                                                    For a (Symmetric) Square Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    Called Eigenvalues

                                                                                    Convenient Ordering

                                                                                    ddX

                                                                                    d

                                                                                    D

                                                                                    0

                                                                                    01

                                                                                    n 1

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition

                                                                                    For a (Symmetric) Square Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    And an Orthonormal Matrix

                                                                                    (ie )

                                                                                    ddX

                                                                                    d

                                                                                    D

                                                                                    0

                                                                                    01

                                                                                    ddB

                                                                                    ddtt IBBBB

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition

                                                                                    For a (Symmetric) Square Matrix

                                                                                    Find a Diagonal Matrix

                                                                                    And an Orthonormal Matrix

                                                                                    (ie )

                                                                                    So that ie

                                                                                    ddX

                                                                                    d

                                                                                    D

                                                                                    0

                                                                                    01

                                                                                    ddB

                                                                                    ddtt IBBBB

                                                                                    DBBX tBDBX

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                    (looks similar)

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                    Complex (uses )

                                                                                    VU

                                                                                    1i

                                                                                    Review of Linear Algebra (Cont)

                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                    VU

                                                                                    X

                                                                                    BVU

                                                                                    1i

                                                                                    • Participant Presentations
                                                                                    • PCA to find clusters
                                                                                    • Statistical Smoothing
                                                                                    • Kernel Density Estimation
                                                                                    • Scatterplot Smoothing
                                                                                    • Statistical Smoothing (2)
                                                                                    • SiZer Background
                                                                                    • SiZer Background (2)
                                                                                    • SiZer Background (3)
                                                                                    • SiZer Background (4)
                                                                                    • SiZer Background (5)
                                                                                    • SiZer Overview
                                                                                    • PCA to find clusters (2)
                                                                                    • PCA to find clusters (3)
                                                                                    • PCA to find clusters (4)
                                                                                    • PCA to find clusters (5)
                                                                                    • PCA to find clusters (6)
                                                                                    • PCA to find clusters (7)
                                                                                    • Recall Yeast Cell Cycle Data
                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                    • Frequency 2 Analysis
                                                                                    • Frequency 2 Analysis (2)
                                                                                    • Frequency 2 Analysis (3)
                                                                                    • Yeast Cell Cycle
                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                    • Reclassification of Major Genes
                                                                                    • Compare to Previous Classifrsquon
                                                                                    • New Subpopulation View
                                                                                    • New Subpopulation View (2)
                                                                                    • Detailed Look at PCA
                                                                                    • PCA Rediscovery ndash Renaming
                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                    • An Interesting Historical Note
                                                                                    • An Interesting Historical Note (2)
                                                                                    • Detailed Look at PCA (2)
                                                                                    • Detailed Look at PCA (3)
                                                                                    • Review of Linear Algebra
                                                                                    • Review of Linear Algebra (2)
                                                                                    • Review of Linear Algebra (3)
                                                                                    • Review of Linear Algebra (Cont)
                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                      Review of Linear Algebra

                                                                                      Vector Space

                                                                                      bull set of ldquovectorsrdquo

                                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                                      xa

                                                                                      Review of Linear Algebra

                                                                                      Vector Space

                                                                                      bull set of ldquovectorsrdquo

                                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                      ( in space)

                                                                                      xa

                                                                                      i

                                                                                      ii xa

                                                                                      Review of Linear Algebra

                                                                                      Vector Space

                                                                                      bull set of ldquovectorsrdquo

                                                                                      bull and ldquoscalarsrdquo (coefficients)

                                                                                      bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                      ( in space)

                                                                                      eg

                                                                                      ldquo dim Euclidrsquon spacerdquo

                                                                                      xa

                                                                                      i

                                                                                      ii xa

                                                                                      d

                                                                                      d

                                                                                      d xx

                                                                                      x

                                                                                      x

                                                                                      x 1

                                                                                      1

                                                                                      d

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                      Note Planes not Through the Origin

                                                                                      are not Subspaces

                                                                                      (Do not Contain )00 x

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                      (all Linear Combos of them =

                                                                                      = Containing Hyperplane

                                                                                      through Origin)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Basis of Subspace Set of Vectors that

                                                                                      bull Span ie Everything is a Lin Com of them

                                                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Basis of Subspace Set of Vectors that

                                                                                      bull Span ie Everything is a Lin Com of them

                                                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                      bull eg ldquoUnit Vector Basisrdquo

                                                                                      bull Since

                                                                                      d

                                                                                      1

                                                                                      0

                                                                                      0

                                                                                      0

                                                                                      1

                                                                                      0

                                                                                      0

                                                                                      0

                                                                                      1

                                                                                      1

                                                                                      0

                                                                                      0

                                                                                      0

                                                                                      1

                                                                                      0

                                                                                      0

                                                                                      0

                                                                                      1

                                                                                      212

                                                                                      1

                                                                                      d

                                                                                      d

                                                                                      xxx

                                                                                      x

                                                                                      x

                                                                                      x

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Basis Matrix of subspace of

                                                                                      Given a basis

                                                                                      create matrix of columns

                                                                                      dnvv 1

                                                                                      nddnd

                                                                                      n

                                                                                      n

                                                                                      vv

                                                                                      vv

                                                                                      vvB

                                                                                      1

                                                                                      111

                                                                                      1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                      where

                                                                                      n

                                                                                      iii aBva

                                                                                      1

                                                                                      na

                                                                                      a

                                                                                      a 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                      where

                                                                                      Note Right Multiplication Gives

                                                                                      Linear Combination of Column Vectors

                                                                                      n

                                                                                      iii aBva

                                                                                      1

                                                                                      na

                                                                                      a

                                                                                      a 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                      where

                                                                                      Check sizes

                                                                                      n

                                                                                      iii aBva

                                                                                      1

                                                                                      na

                                                                                      a

                                                                                      a 1

                                                                                      )1()(1 nndd

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                      For matrices

                                                                                      Define the Matrix Product

                                                                                      mkk

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      nmm

                                                                                      n

                                                                                      bb

                                                                                      bb

                                                                                      B

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iniik

                                                                                      m

                                                                                      iiik

                                                                                      m

                                                                                      inii

                                                                                      m

                                                                                      iii

                                                                                      baba

                                                                                      baba

                                                                                      AB

                                                                                      1

                                                                                      11

                                                                                      11

                                                                                      111

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                      For matrices

                                                                                      Define the Matrix Product

                                                                                      (Inner Products of Rows With Columns )

                                                                                      mkk

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      nmm

                                                                                      n

                                                                                      bb

                                                                                      bb

                                                                                      B

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iniik

                                                                                      m

                                                                                      iiik

                                                                                      m

                                                                                      inii

                                                                                      m

                                                                                      iii

                                                                                      baba

                                                                                      baba

                                                                                      AB

                                                                                      1

                                                                                      11

                                                                                      11

                                                                                      111

                                                                                      A B

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                      For matrices

                                                                                      Define the Matrix Product

                                                                                      (Inner Products of Rows With Columns )

                                                                                      (Composition of Linear Transformations)

                                                                                      mkk

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      nmm

                                                                                      n

                                                                                      bb

                                                                                      bb

                                                                                      B

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iniik

                                                                                      m

                                                                                      iiik

                                                                                      m

                                                                                      inii

                                                                                      m

                                                                                      iii

                                                                                      baba

                                                                                      baba

                                                                                      AB

                                                                                      1

                                                                                      11

                                                                                      11

                                                                                      111

                                                                                      A B

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                      For matrices

                                                                                      Define the Matrix Product

                                                                                      (Inner Products of Rows With Columns )

                                                                                      (Composition of Linear Transformations)

                                                                                      Often Useful to Check Sizes

                                                                                      mkk

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      nmm

                                                                                      n

                                                                                      bb

                                                                                      bb

                                                                                      B

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iniik

                                                                                      m

                                                                                      iiik

                                                                                      m

                                                                                      inii

                                                                                      m

                                                                                      iii

                                                                                      baba

                                                                                      baba

                                                                                      AB

                                                                                      1

                                                                                      11

                                                                                      11

                                                                                      111

                                                                                      nmmknk

                                                                                      A B

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                      For matrices

                                                                                      Define the Matrix Product

                                                                                      (Inner Products of Rows With Columns )

                                                                                      (Composition of Linear Transformations)

                                                                                      Often Useful to Check Sizes

                                                                                      mkk

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      nmm

                                                                                      n

                                                                                      bb

                                                                                      bb

                                                                                      B

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iniik

                                                                                      m

                                                                                      iiik

                                                                                      m

                                                                                      inii

                                                                                      m

                                                                                      iii

                                                                                      baba

                                                                                      baba

                                                                                      AB

                                                                                      1

                                                                                      11

                                                                                      11

                                                                                      111

                                                                                      nmmknk

                                                                                      A B

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Matrix Trace

                                                                                      bull For a Square Matrix

                                                                                      bull Define

                                                                                      mmm

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iiiaAtr

                                                                                      1)(

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Matrix Trace

                                                                                      bull For a Square Matrix

                                                                                      bull Define

                                                                                      bull Trace Commutes with Matrix Multiplication

                                                                                      mmm

                                                                                      m

                                                                                      aa

                                                                                      aa

                                                                                      A

                                                                                      1

                                                                                      111

                                                                                      m

                                                                                      iiiaAtr

                                                                                      1)(

                                                                                      BAtrABtr

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                      bull (Use Basis Above)

                                                                                      bull eg dim of a line is 1

                                                                                      bull eg dim of a plane is 2

                                                                                      dd dim

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                      bull (Use Basis Above)

                                                                                      bull eg dim of a line is 1

                                                                                      bull eg dim of a plane is 2

                                                                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                      (in Statistical Uses eg ANOVA)

                                                                                      dd dim

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Norm of a Vector

                                                                                      bull in d 21

                                                                                      21

                                                                                      1

                                                                                      2 xxxx td

                                                                                      jj

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Norm of a Vector

                                                                                      bull in

                                                                                      bull Idea length of the vector

                                                                                      d 2121

                                                                                      1

                                                                                      2 xxxx td

                                                                                      jj

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Norm of a Vector

                                                                                      bull in

                                                                                      bull Idea length of the vector

                                                                                      bull Note strange properties for high

                                                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                                                      d 2121

                                                                                      1

                                                                                      2 xxxx td

                                                                                      jj

                                                                                      d

                                                                                      d

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Norm of a Vector (cont)

                                                                                      bull Length Normalized Vector

                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                      amp is a Direction Vector)

                                                                                      x

                                                                                      x

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Norm of a Vector (cont)

                                                                                      bull Length Normalized Vector

                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                      amp is a Direction Vector)

                                                                                      bull Define Distance as

                                                                                      x

                                                                                      x

                                                                                      yxyxyxyxd t

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Inner (Dot Scalar) Product

                                                                                      bull for Vectors and

                                                                                      yxyxyx td

                                                                                      jjj

                                                                                      1

                                                                                      x y

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Inner (Dot Scalar) Product

                                                                                      bull for Vectors and

                                                                                      bull Related to Norm via

                                                                                      yxyxyx td

                                                                                      jjj

                                                                                      1

                                                                                      x y

                                                                                      21 xxx

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                      bull measures ldquoangle between and rdquo as

                                                                                      yyxx

                                                                                      yx

                                                                                      yx

                                                                                      yxyxangle

                                                                                      tt

                                                                                      t

                                                                                      11 cos

                                                                                      cos

                                                                                      x y

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                      bull measures ldquoangle between and rdquo as

                                                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                                                      if and only if

                                                                                      yyxx

                                                                                      yx

                                                                                      yx

                                                                                      yxyxangle

                                                                                      tt

                                                                                      t

                                                                                      11 cos

                                                                                      cos

                                                                                      x y

                                                                                      yx 0 yx

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis

                                                                                      bull All Orthogonal to each other

                                                                                      ie for

                                                                                      bull All have Length 1

                                                                                      ie for

                                                                                      nvv 1

                                                                                      1 ii vv

                                                                                      0 ii vv ii

                                                                                      ni 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis (cont)

                                                                                      bull Spectral Representation

                                                                                      where

                                                                                      nvv 1

                                                                                      n

                                                                                      iii vax

                                                                                      1

                                                                                      ii vxa

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis (cont)

                                                                                      bull Spectral Representation

                                                                                      where

                                                                                      (Coefficient is Inner Product Cool Notation)

                                                                                      nvv 1

                                                                                      n

                                                                                      iii vax

                                                                                      1

                                                                                      ii vxa

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis (cont)

                                                                                      bull Spectral Representation

                                                                                      where

                                                                                      Check

                                                                                      nvv 1

                                                                                      n

                                                                                      iii vax

                                                                                      1

                                                                                      ii vxa

                                                                                      iii

                                                                                      n

                                                                                      iii

                                                                                      n

                                                                                      iiii avvavvavx

                                                                                      1

                                                                                      1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis (cont)

                                                                                      bull Spectral Representation

                                                                                      where

                                                                                      Check

                                                                                      bull Matrix Notation where ie

                                                                                      For the Basis Matrix

                                                                                      nvv 1

                                                                                      n

                                                                                      iii vax

                                                                                      1

                                                                                      ii vxa

                                                                                      iii

                                                                                      n

                                                                                      iii

                                                                                      n

                                                                                      iiii avvavvavx

                                                                                      1

                                                                                      1

                                                                                      aBx Bxa tt xBa t

                                                                                      nvvB 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Orthonormal Basis (cont)

                                                                                      bull Spectral Representation

                                                                                      where

                                                                                      Check

                                                                                      bull Matrix Notation where ie

                                                                                      is called transform of

                                                                                      (eg Fourier or Wavelet)

                                                                                      nvv 1

                                                                                      n

                                                                                      iii vax

                                                                                      1

                                                                                      ii vxa

                                                                                      iii

                                                                                      n

                                                                                      iii

                                                                                      n

                                                                                      iiii avvavvavx

                                                                                      1

                                                                                      1

                                                                                      aBx Bxa tt xBa t

                                                                                      xa

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Parseval identity for

                                                                                      in subsp genrsquod by o n basis

                                                                                      x

                                                                                      nvv 1

                                                                                      2

                                                                                      1

                                                                                      22

                                                                                      1

                                                                                      2 aavxx

                                                                                      n

                                                                                      ii

                                                                                      n

                                                                                      ii

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Parseval identity for

                                                                                      in subsp genrsquod by o n basis

                                                                                      bull Pythagorean theorem

                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                      bull ANOVA - sums of squares

                                                                                      x

                                                                                      nvv 1

                                                                                      2

                                                                                      1

                                                                                      22

                                                                                      1

                                                                                      2 aavxx

                                                                                      n

                                                                                      ii

                                                                                      n

                                                                                      ii

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Parseval identity for

                                                                                      in subsp genrsquod by o n basis

                                                                                      bull Pythagorean theorem

                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                      bull ANOVA - sums of squares

                                                                                      bull Transform has same length as

                                                                                      ie ldquorotation in rdquo

                                                                                      x

                                                                                      nvv 1

                                                                                      2

                                                                                      1

                                                                                      22

                                                                                      1

                                                                                      2 aavxx

                                                                                      n

                                                                                      ii

                                                                                      n

                                                                                      ii

                                                                                      a xd

                                                                                      Projection of a Vector onto a Subspace

                                                                                      bull Idea Member of that is Closest to

                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                      Review of Linear Algebra (Cont)x

                                                                                      xV

                                                                                      V

                                                                                      Projection of a Vector onto a Subspace

                                                                                      bull Idea Member of that is Closest to

                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                      bull Find that Solves

                                                                                      (ldquoLeast Squaresrdquo)

                                                                                      Review of Linear Algebra (Cont)x

                                                                                      xV

                                                                                      V

                                                                                      VxPV vxVv

                                                                                      min

                                                                                      Projection of a Vector onto a Subspace

                                                                                      bull Idea Member of that is Closest to

                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                      bull Find that Solves

                                                                                      (ldquoLeast Squaresrdquo)

                                                                                      bull For Inner Product (Hilbert) Space

                                                                                      Exists and is Unique

                                                                                      Review of Linear Algebra (Cont)x

                                                                                      xV

                                                                                      V

                                                                                      VxPV vxVv

                                                                                      min

                                                                                      xPV

                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                      bull General Solution in for Basis Matrix

                                                                                      Review of Linear Algebra (Cont)

                                                                                      d VB

                                                                                      xBBBBxP tVV

                                                                                      tVVV

                                                                                      1

                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                      bull General Solution in for Basis Matrix

                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                      (thus projection is another linear operation)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      d VB

                                                                                      xBBBBxP tVV

                                                                                      tVVV

                                                                                      1

                                                                                      tVV

                                                                                      tVVV BBBBP

                                                                                      1

                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                      bull General Solution in for Basis Matrix

                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                      (thus projection is another linear operation)

                                                                                      (note same operation underlies least squares)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      d VB

                                                                                      xBBBBxP tVV

                                                                                      tVVV

                                                                                      1

                                                                                      tVV

                                                                                      tVVV BBBBP

                                                                                      1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Projection using Orthonormal Basis

                                                                                      bull Basis Matrix is Orthonormal

                                                                                      nnVtV IBB

                                                                                      10

                                                                                      01

                                                                                      1

                                                                                      111

                                                                                      1

                                                                                      1

                                                                                      nnn

                                                                                      n

                                                                                      ntn

                                                                                      t

                                                                                      vvvv

                                                                                      vvvv

                                                                                      vv

                                                                                      v

                                                                                      v

                                                                                      nvv 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Projection using Orthonormal Basis

                                                                                      bull Basis Matrix is Orthonormal

                                                                                      bull So =

                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                      (Recall Right Multrsquon)

                                                                                      nnVtV IBB

                                                                                      10

                                                                                      01

                                                                                      1

                                                                                      111

                                                                                      1

                                                                                      1

                                                                                      nnn

                                                                                      n

                                                                                      ntn

                                                                                      t

                                                                                      vvvv

                                                                                      vvvv

                                                                                      vv

                                                                                      v

                                                                                      v

                                                                                      xBBxP tVVV

                                                                                      x V

                                                                                      nvv 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Projection using Orthonormal Basis (cont)

                                                                                      bull For Orthogonal Complement

                                                                                      and

                                                                                      V

                                                                                      xPxPx VV 222xPxPx VV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Projection using Orthonormal Basis (cont)

                                                                                      bull For Orthogonal Complement

                                                                                      and

                                                                                      bull Parseval Inequality

                                                                                      V

                                                                                      xPxPx VV 222xPxPx VV

                                                                                      2

                                                                                      1

                                                                                      22

                                                                                      1

                                                                                      22 aavxxxP

                                                                                      n

                                                                                      ii

                                                                                      n

                                                                                      iiV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      (Real) Unitary Matrices with

                                                                                      bull Orthonormal Basis Matrix

                                                                                      (So All of Above Applies)

                                                                                      ddU IUU t

                                                                                      Review of Linear Algebra (Cont)

                                                                                      (Real) Unitary Matrices with

                                                                                      bull Orthonormal Basis Matrix

                                                                                      (So All of Above Applies)

                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                      ddU IUU t

                                                                                      yxdyxyxyxUyUxUdn

                                                                                      i ii 2

                                                                                      1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      (Real) Unitary Matrices with

                                                                                      bull Orthonormal Basis Matrix

                                                                                      (So All of Above Applies)

                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                      ddU IUU t

                                                                                      yxdyxyxyxUyUxUdn

                                                                                      i ii 2

                                                                                      1

                                                                                      U

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Singular Value Decomposition (SVD)

                                                                                      For a Matrix

                                                                                      Find

                                                                                      ndX

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Singular Value Decomposition (SVD)

                                                                                      For a Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      with Entries

                                                                                      called Singular Values

                                                                                      ndX

                                                                                      ndS

                                                                                      )min(1 ndss

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Singular Value Decomposition (SVD)

                                                                                      For a Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      with Entries

                                                                                      called Singular Values

                                                                                      And Unitary (Rotation) Matrices

                                                                                      (recall )

                                                                                      ndX

                                                                                      ndS

                                                                                      )min(1 ndss

                                                                                      ddU nnV

                                                                                      IVVUU tt

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Singular Value Decomposition (SVD)

                                                                                      For a Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      with Entries

                                                                                      called Singular Values

                                                                                      And Unitary (Rotation) Matrices

                                                                                      (recall )

                                                                                      So That

                                                                                      ndX

                                                                                      ndS

                                                                                      )min(1 ndss

                                                                                      ddU nnV

                                                                                      IVVUU tt tUSVX

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Intuition behind Singular Value Decomposition

                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                      vVSUvVSUvX tt

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Intuition behind Singular Value Decomposition

                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                      bull First rotate

                                                                                      X

                                                                                      vVSUvVSUvX tt

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Intuition behind Singular Value Decomposition

                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                      bull First rotate

                                                                                      bull Second rescale coordinate axes (by )

                                                                                      X

                                                                                      vVSUvVSUvX tt

                                                                                      is

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Intuition behind Singular Value Decomposition

                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                      bull First rotate

                                                                                      bull Second rescale coordinate axes (by )

                                                                                      bull Third rotate again

                                                                                      X

                                                                                      vVSUvVSUvX tt

                                                                                      is

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Intuition behind Singular Value Decomposition

                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                      bull First rotate

                                                                                      bull Second rescale coordinate axes (by )

                                                                                      bull Third rotate again

                                                                                      bull ie have diagonalized the transformation

                                                                                      X

                                                                                      vVSUvVSUvX tt

                                                                                      is

                                                                                      SVD Compact Representation

                                                                                      Useful Labeling

                                                                                      Singular Values in Increasing Order

                                                                                      Review of Linear Algebra (Cont)

                                                                                      )min(1 dnss

                                                                                      SVD Compact Representation

                                                                                      Useful Labeling

                                                                                      Singular Values in Increasing Order

                                                                                      Note singular values = 0 can be omitted

                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      )min(1 dnss

                                                                                      Review of Linear Algebra (Cont)

                                                                                      r

                                                                                      SVD Compact Representation

                                                                                      Useful Labeling

                                                                                      Singular Values in Increasing Order

                                                                                      Note singular values = 0 can be omitted

                                                                                      Let = of positive singular values

                                                                                      )min(1 dnss

                                                                                      Review of Linear Algebra (Cont)

                                                                                      r

                                                                                      SVD Compact Representation

                                                                                      Useful Labeling

                                                                                      Singular Values in Increasing Order

                                                                                      Note singular values = 0 can be omitted

                                                                                      Let = of positive singular values

                                                                                      Then

                                                                                      Where are truncations of

                                                                                      trnrrrd VSUX

                                                                                      VSU

                                                                                      )min(1 dnss

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Full Representation

                                                                                      =

                                                                                      Graphics Display Assumes

                                                                                      ndX ddU ndS nn

                                                                                      tV

                                                                                      nd

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Full Representation

                                                                                      =

                                                                                      Full Rank Basis Matrix

                                                                                      ndX ddU ndS nn

                                                                                      tV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Full Representation

                                                                                      =

                                                                                      Full Rank Basis Matrix

                                                                                      All 0s in Bottom

                                                                                      ndX ddU ndS nn

                                                                                      tV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Reduced Representation

                                                                                      =

                                                                                      These Columns Get 0ed Out

                                                                                      ndX ddU nnS nn

                                                                                      tV

                                                                                      nnd 0

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Reduced Representation

                                                                                      =ndX ndU nnS nn

                                                                                      tV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Reduced Representation

                                                                                      =

                                                                                      Also Some of These May be 0

                                                                                      ndX ndU nnS nn

                                                                                      tV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Compact Representation

                                                                                      = ndX rdU

                                                                                      rrS nrtV

                                                                                      0

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Compact Representation

                                                                                      =

                                                                                      These Get 0ed Out

                                                                                      ndX rdU

                                                                                      rrS nrtV

                                                                                      0

                                                                                      Review of Linear Algebra (Cont)

                                                                                      SVD Compact Representation

                                                                                      = ndX rdU

                                                                                      rrS nrtV

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition

                                                                                      For a (Symmetric) Square Matrix

                                                                                      Find

                                                                                      ddX

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition

                                                                                      For a (Symmetric) Square Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      Called Eigenvalues

                                                                                      Convenient Ordering

                                                                                      ddX

                                                                                      d

                                                                                      D

                                                                                      0

                                                                                      01

                                                                                      n 1

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition

                                                                                      For a (Symmetric) Square Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      And an Orthonormal Matrix

                                                                                      (ie )

                                                                                      ddX

                                                                                      d

                                                                                      D

                                                                                      0

                                                                                      01

                                                                                      ddB

                                                                                      ddtt IBBBB

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition

                                                                                      For a (Symmetric) Square Matrix

                                                                                      Find a Diagonal Matrix

                                                                                      And an Orthonormal Matrix

                                                                                      (ie )

                                                                                      So that ie

                                                                                      ddX

                                                                                      d

                                                                                      D

                                                                                      0

                                                                                      01

                                                                                      ddB

                                                                                      ddtt IBBBB

                                                                                      DBBX tBDBX

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                      (looks similar)

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                      Complex (uses )

                                                                                      VU

                                                                                      1i

                                                                                      Review of Linear Algebra (Cont)

                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                      VU

                                                                                      X

                                                                                      BVU

                                                                                      1i

                                                                                      • Participant Presentations
                                                                                      • PCA to find clusters
                                                                                      • Statistical Smoothing
                                                                                      • Kernel Density Estimation
                                                                                      • Scatterplot Smoothing
                                                                                      • Statistical Smoothing (2)
                                                                                      • SiZer Background
                                                                                      • SiZer Background (2)
                                                                                      • SiZer Background (3)
                                                                                      • SiZer Background (4)
                                                                                      • SiZer Background (5)
                                                                                      • SiZer Overview
                                                                                      • PCA to find clusters (2)
                                                                                      • PCA to find clusters (3)
                                                                                      • PCA to find clusters (4)
                                                                                      • PCA to find clusters (5)
                                                                                      • PCA to find clusters (6)
                                                                                      • PCA to find clusters (7)
                                                                                      • Recall Yeast Cell Cycle Data
                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                      • Frequency 2 Analysis
                                                                                      • Frequency 2 Analysis (2)
                                                                                      • Frequency 2 Analysis (3)
                                                                                      • Yeast Cell Cycle
                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                      • Reclassification of Major Genes
                                                                                      • Compare to Previous Classifrsquon
                                                                                      • New Subpopulation View
                                                                                      • New Subpopulation View (2)
                                                                                      • Detailed Look at PCA
                                                                                      • PCA Rediscovery ndash Renaming
                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                      • An Interesting Historical Note
                                                                                      • An Interesting Historical Note (2)
                                                                                      • Detailed Look at PCA (2)
                                                                                      • Detailed Look at PCA (3)
                                                                                      • Review of Linear Algebra
                                                                                      • Review of Linear Algebra (2)
                                                                                      • Review of Linear Algebra (3)
                                                                                      • Review of Linear Algebra (Cont)
                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                        Review of Linear Algebra

                                                                                        Vector Space

                                                                                        bull set of ldquovectorsrdquo

                                                                                        bull and ldquoscalarsrdquo (coefficients)

                                                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                        ( in space)

                                                                                        xa

                                                                                        i

                                                                                        ii xa

                                                                                        Review of Linear Algebra

                                                                                        Vector Space

                                                                                        bull set of ldquovectorsrdquo

                                                                                        bull and ldquoscalarsrdquo (coefficients)

                                                                                        bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                        ( in space)

                                                                                        eg

                                                                                        ldquo dim Euclidrsquon spacerdquo

                                                                                        xa

                                                                                        i

                                                                                        ii xa

                                                                                        d

                                                                                        d

                                                                                        d xx

                                                                                        x

                                                                                        x

                                                                                        x 1

                                                                                        1

                                                                                        d

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                        Note Planes not Through the Origin

                                                                                        are not Subspaces

                                                                                        (Do not Contain )00 x

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                        (all Linear Combos of them =

                                                                                        = Containing Hyperplane

                                                                                        through Origin)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Basis of Subspace Set of Vectors that

                                                                                        bull Span ie Everything is a Lin Com of them

                                                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Basis of Subspace Set of Vectors that

                                                                                        bull Span ie Everything is a Lin Com of them

                                                                                        bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                        bull eg ldquoUnit Vector Basisrdquo

                                                                                        bull Since

                                                                                        d

                                                                                        1

                                                                                        0

                                                                                        0

                                                                                        0

                                                                                        1

                                                                                        0

                                                                                        0

                                                                                        0

                                                                                        1

                                                                                        1

                                                                                        0

                                                                                        0

                                                                                        0

                                                                                        1

                                                                                        0

                                                                                        0

                                                                                        0

                                                                                        1

                                                                                        212

                                                                                        1

                                                                                        d

                                                                                        d

                                                                                        xxx

                                                                                        x

                                                                                        x

                                                                                        x

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Basis Matrix of subspace of

                                                                                        Given a basis

                                                                                        create matrix of columns

                                                                                        dnvv 1

                                                                                        nddnd

                                                                                        n

                                                                                        n

                                                                                        vv

                                                                                        vv

                                                                                        vvB

                                                                                        1

                                                                                        111

                                                                                        1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                        where

                                                                                        n

                                                                                        iii aBva

                                                                                        1

                                                                                        na

                                                                                        a

                                                                                        a 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                        where

                                                                                        Note Right Multiplication Gives

                                                                                        Linear Combination of Column Vectors

                                                                                        n

                                                                                        iii aBva

                                                                                        1

                                                                                        na

                                                                                        a

                                                                                        a 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                        where

                                                                                        Check sizes

                                                                                        n

                                                                                        iii aBva

                                                                                        1

                                                                                        na

                                                                                        a

                                                                                        a 1

                                                                                        )1()(1 nndd

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                        For matrices

                                                                                        Define the Matrix Product

                                                                                        mkk

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        nmm

                                                                                        n

                                                                                        bb

                                                                                        bb

                                                                                        B

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iniik

                                                                                        m

                                                                                        iiik

                                                                                        m

                                                                                        inii

                                                                                        m

                                                                                        iii

                                                                                        baba

                                                                                        baba

                                                                                        AB

                                                                                        1

                                                                                        11

                                                                                        11

                                                                                        111

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                        For matrices

                                                                                        Define the Matrix Product

                                                                                        (Inner Products of Rows With Columns )

                                                                                        mkk

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        nmm

                                                                                        n

                                                                                        bb

                                                                                        bb

                                                                                        B

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iniik

                                                                                        m

                                                                                        iiik

                                                                                        m

                                                                                        inii

                                                                                        m

                                                                                        iii

                                                                                        baba

                                                                                        baba

                                                                                        AB

                                                                                        1

                                                                                        11

                                                                                        11

                                                                                        111

                                                                                        A B

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                        For matrices

                                                                                        Define the Matrix Product

                                                                                        (Inner Products of Rows With Columns )

                                                                                        (Composition of Linear Transformations)

                                                                                        mkk

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        nmm

                                                                                        n

                                                                                        bb

                                                                                        bb

                                                                                        B

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iniik

                                                                                        m

                                                                                        iiik

                                                                                        m

                                                                                        inii

                                                                                        m

                                                                                        iii

                                                                                        baba

                                                                                        baba

                                                                                        AB

                                                                                        1

                                                                                        11

                                                                                        11

                                                                                        111

                                                                                        A B

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                        For matrices

                                                                                        Define the Matrix Product

                                                                                        (Inner Products of Rows With Columns )

                                                                                        (Composition of Linear Transformations)

                                                                                        Often Useful to Check Sizes

                                                                                        mkk

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        nmm

                                                                                        n

                                                                                        bb

                                                                                        bb

                                                                                        B

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iniik

                                                                                        m

                                                                                        iiik

                                                                                        m

                                                                                        inii

                                                                                        m

                                                                                        iii

                                                                                        baba

                                                                                        baba

                                                                                        AB

                                                                                        1

                                                                                        11

                                                                                        11

                                                                                        111

                                                                                        nmmknk

                                                                                        A B

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                        For matrices

                                                                                        Define the Matrix Product

                                                                                        (Inner Products of Rows With Columns )

                                                                                        (Composition of Linear Transformations)

                                                                                        Often Useful to Check Sizes

                                                                                        mkk

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        nmm

                                                                                        n

                                                                                        bb

                                                                                        bb

                                                                                        B

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iniik

                                                                                        m

                                                                                        iiik

                                                                                        m

                                                                                        inii

                                                                                        m

                                                                                        iii

                                                                                        baba

                                                                                        baba

                                                                                        AB

                                                                                        1

                                                                                        11

                                                                                        11

                                                                                        111

                                                                                        nmmknk

                                                                                        A B

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Matrix Trace

                                                                                        bull For a Square Matrix

                                                                                        bull Define

                                                                                        mmm

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iiiaAtr

                                                                                        1)(

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Matrix Trace

                                                                                        bull For a Square Matrix

                                                                                        bull Define

                                                                                        bull Trace Commutes with Matrix Multiplication

                                                                                        mmm

                                                                                        m

                                                                                        aa

                                                                                        aa

                                                                                        A

                                                                                        1

                                                                                        111

                                                                                        m

                                                                                        iiiaAtr

                                                                                        1)(

                                                                                        BAtrABtr

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                        bull (Use Basis Above)

                                                                                        bull eg dim of a line is 1

                                                                                        bull eg dim of a plane is 2

                                                                                        dd dim

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                        bull (Use Basis Above)

                                                                                        bull eg dim of a line is 1

                                                                                        bull eg dim of a plane is 2

                                                                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                        (in Statistical Uses eg ANOVA)

                                                                                        dd dim

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Norm of a Vector

                                                                                        bull in d 21

                                                                                        21

                                                                                        1

                                                                                        2 xxxx td

                                                                                        jj

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Norm of a Vector

                                                                                        bull in

                                                                                        bull Idea length of the vector

                                                                                        d 2121

                                                                                        1

                                                                                        2 xxxx td

                                                                                        jj

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Norm of a Vector

                                                                                        bull in

                                                                                        bull Idea length of the vector

                                                                                        bull Note strange properties for high

                                                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                                                        d 2121

                                                                                        1

                                                                                        2 xxxx td

                                                                                        jj

                                                                                        d

                                                                                        d

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Norm of a Vector (cont)

                                                                                        bull Length Normalized Vector

                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                        amp is a Direction Vector)

                                                                                        x

                                                                                        x

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Norm of a Vector (cont)

                                                                                        bull Length Normalized Vector

                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                        amp is a Direction Vector)

                                                                                        bull Define Distance as

                                                                                        x

                                                                                        x

                                                                                        yxyxyxyxd t

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Inner (Dot Scalar) Product

                                                                                        bull for Vectors and

                                                                                        yxyxyx td

                                                                                        jjj

                                                                                        1

                                                                                        x y

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Inner (Dot Scalar) Product

                                                                                        bull for Vectors and

                                                                                        bull Related to Norm via

                                                                                        yxyxyx td

                                                                                        jjj

                                                                                        1

                                                                                        x y

                                                                                        21 xxx

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                        bull measures ldquoangle between and rdquo as

                                                                                        yyxx

                                                                                        yx

                                                                                        yx

                                                                                        yxyxangle

                                                                                        tt

                                                                                        t

                                                                                        11 cos

                                                                                        cos

                                                                                        x y

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                        bull measures ldquoangle between and rdquo as

                                                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                                                        if and only if

                                                                                        yyxx

                                                                                        yx

                                                                                        yx

                                                                                        yxyxangle

                                                                                        tt

                                                                                        t

                                                                                        11 cos

                                                                                        cos

                                                                                        x y

                                                                                        yx 0 yx

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis

                                                                                        bull All Orthogonal to each other

                                                                                        ie for

                                                                                        bull All have Length 1

                                                                                        ie for

                                                                                        nvv 1

                                                                                        1 ii vv

                                                                                        0 ii vv ii

                                                                                        ni 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis (cont)

                                                                                        bull Spectral Representation

                                                                                        where

                                                                                        nvv 1

                                                                                        n

                                                                                        iii vax

                                                                                        1

                                                                                        ii vxa

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis (cont)

                                                                                        bull Spectral Representation

                                                                                        where

                                                                                        (Coefficient is Inner Product Cool Notation)

                                                                                        nvv 1

                                                                                        n

                                                                                        iii vax

                                                                                        1

                                                                                        ii vxa

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis (cont)

                                                                                        bull Spectral Representation

                                                                                        where

                                                                                        Check

                                                                                        nvv 1

                                                                                        n

                                                                                        iii vax

                                                                                        1

                                                                                        ii vxa

                                                                                        iii

                                                                                        n

                                                                                        iii

                                                                                        n

                                                                                        iiii avvavvavx

                                                                                        1

                                                                                        1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis (cont)

                                                                                        bull Spectral Representation

                                                                                        where

                                                                                        Check

                                                                                        bull Matrix Notation where ie

                                                                                        For the Basis Matrix

                                                                                        nvv 1

                                                                                        n

                                                                                        iii vax

                                                                                        1

                                                                                        ii vxa

                                                                                        iii

                                                                                        n

                                                                                        iii

                                                                                        n

                                                                                        iiii avvavvavx

                                                                                        1

                                                                                        1

                                                                                        aBx Bxa tt xBa t

                                                                                        nvvB 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Orthonormal Basis (cont)

                                                                                        bull Spectral Representation

                                                                                        where

                                                                                        Check

                                                                                        bull Matrix Notation where ie

                                                                                        is called transform of

                                                                                        (eg Fourier or Wavelet)

                                                                                        nvv 1

                                                                                        n

                                                                                        iii vax

                                                                                        1

                                                                                        ii vxa

                                                                                        iii

                                                                                        n

                                                                                        iii

                                                                                        n

                                                                                        iiii avvavvavx

                                                                                        1

                                                                                        1

                                                                                        aBx Bxa tt xBa t

                                                                                        xa

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Parseval identity for

                                                                                        in subsp genrsquod by o n basis

                                                                                        x

                                                                                        nvv 1

                                                                                        2

                                                                                        1

                                                                                        22

                                                                                        1

                                                                                        2 aavxx

                                                                                        n

                                                                                        ii

                                                                                        n

                                                                                        ii

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Parseval identity for

                                                                                        in subsp genrsquod by o n basis

                                                                                        bull Pythagorean theorem

                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                        bull ANOVA - sums of squares

                                                                                        x

                                                                                        nvv 1

                                                                                        2

                                                                                        1

                                                                                        22

                                                                                        1

                                                                                        2 aavxx

                                                                                        n

                                                                                        ii

                                                                                        n

                                                                                        ii

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Parseval identity for

                                                                                        in subsp genrsquod by o n basis

                                                                                        bull Pythagorean theorem

                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                        bull ANOVA - sums of squares

                                                                                        bull Transform has same length as

                                                                                        ie ldquorotation in rdquo

                                                                                        x

                                                                                        nvv 1

                                                                                        2

                                                                                        1

                                                                                        22

                                                                                        1

                                                                                        2 aavxx

                                                                                        n

                                                                                        ii

                                                                                        n

                                                                                        ii

                                                                                        a xd

                                                                                        Projection of a Vector onto a Subspace

                                                                                        bull Idea Member of that is Closest to

                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                        Review of Linear Algebra (Cont)x

                                                                                        xV

                                                                                        V

                                                                                        Projection of a Vector onto a Subspace

                                                                                        bull Idea Member of that is Closest to

                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                        bull Find that Solves

                                                                                        (ldquoLeast Squaresrdquo)

                                                                                        Review of Linear Algebra (Cont)x

                                                                                        xV

                                                                                        V

                                                                                        VxPV vxVv

                                                                                        min

                                                                                        Projection of a Vector onto a Subspace

                                                                                        bull Idea Member of that is Closest to

                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                        bull Find that Solves

                                                                                        (ldquoLeast Squaresrdquo)

                                                                                        bull For Inner Product (Hilbert) Space

                                                                                        Exists and is Unique

                                                                                        Review of Linear Algebra (Cont)x

                                                                                        xV

                                                                                        V

                                                                                        VxPV vxVv

                                                                                        min

                                                                                        xPV

                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                        bull General Solution in for Basis Matrix

                                                                                        Review of Linear Algebra (Cont)

                                                                                        d VB

                                                                                        xBBBBxP tVV

                                                                                        tVVV

                                                                                        1

                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                        bull General Solution in for Basis Matrix

                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                        (thus projection is another linear operation)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        d VB

                                                                                        xBBBBxP tVV

                                                                                        tVVV

                                                                                        1

                                                                                        tVV

                                                                                        tVVV BBBBP

                                                                                        1

                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                        bull General Solution in for Basis Matrix

                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                        (thus projection is another linear operation)

                                                                                        (note same operation underlies least squares)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        d VB

                                                                                        xBBBBxP tVV

                                                                                        tVVV

                                                                                        1

                                                                                        tVV

                                                                                        tVVV BBBBP

                                                                                        1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Projection using Orthonormal Basis

                                                                                        bull Basis Matrix is Orthonormal

                                                                                        nnVtV IBB

                                                                                        10

                                                                                        01

                                                                                        1

                                                                                        111

                                                                                        1

                                                                                        1

                                                                                        nnn

                                                                                        n

                                                                                        ntn

                                                                                        t

                                                                                        vvvv

                                                                                        vvvv

                                                                                        vv

                                                                                        v

                                                                                        v

                                                                                        nvv 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Projection using Orthonormal Basis

                                                                                        bull Basis Matrix is Orthonormal

                                                                                        bull So =

                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                        (Recall Right Multrsquon)

                                                                                        nnVtV IBB

                                                                                        10

                                                                                        01

                                                                                        1

                                                                                        111

                                                                                        1

                                                                                        1

                                                                                        nnn

                                                                                        n

                                                                                        ntn

                                                                                        t

                                                                                        vvvv

                                                                                        vvvv

                                                                                        vv

                                                                                        v

                                                                                        v

                                                                                        xBBxP tVVV

                                                                                        x V

                                                                                        nvv 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Projection using Orthonormal Basis (cont)

                                                                                        bull For Orthogonal Complement

                                                                                        and

                                                                                        V

                                                                                        xPxPx VV 222xPxPx VV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Projection using Orthonormal Basis (cont)

                                                                                        bull For Orthogonal Complement

                                                                                        and

                                                                                        bull Parseval Inequality

                                                                                        V

                                                                                        xPxPx VV 222xPxPx VV

                                                                                        2

                                                                                        1

                                                                                        22

                                                                                        1

                                                                                        22 aavxxxP

                                                                                        n

                                                                                        ii

                                                                                        n

                                                                                        iiV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        (Real) Unitary Matrices with

                                                                                        bull Orthonormal Basis Matrix

                                                                                        (So All of Above Applies)

                                                                                        ddU IUU t

                                                                                        Review of Linear Algebra (Cont)

                                                                                        (Real) Unitary Matrices with

                                                                                        bull Orthonormal Basis Matrix

                                                                                        (So All of Above Applies)

                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                        ddU IUU t

                                                                                        yxdyxyxyxUyUxUdn

                                                                                        i ii 2

                                                                                        1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        (Real) Unitary Matrices with

                                                                                        bull Orthonormal Basis Matrix

                                                                                        (So All of Above Applies)

                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                        ddU IUU t

                                                                                        yxdyxyxyxUyUxUdn

                                                                                        i ii 2

                                                                                        1

                                                                                        U

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Singular Value Decomposition (SVD)

                                                                                        For a Matrix

                                                                                        Find

                                                                                        ndX

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Singular Value Decomposition (SVD)

                                                                                        For a Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        with Entries

                                                                                        called Singular Values

                                                                                        ndX

                                                                                        ndS

                                                                                        )min(1 ndss

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Singular Value Decomposition (SVD)

                                                                                        For a Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        with Entries

                                                                                        called Singular Values

                                                                                        And Unitary (Rotation) Matrices

                                                                                        (recall )

                                                                                        ndX

                                                                                        ndS

                                                                                        )min(1 ndss

                                                                                        ddU nnV

                                                                                        IVVUU tt

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Singular Value Decomposition (SVD)

                                                                                        For a Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        with Entries

                                                                                        called Singular Values

                                                                                        And Unitary (Rotation) Matrices

                                                                                        (recall )

                                                                                        So That

                                                                                        ndX

                                                                                        ndS

                                                                                        )min(1 ndss

                                                                                        ddU nnV

                                                                                        IVVUU tt tUSVX

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Intuition behind Singular Value Decomposition

                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                        vVSUvVSUvX tt

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Intuition behind Singular Value Decomposition

                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                        bull First rotate

                                                                                        X

                                                                                        vVSUvVSUvX tt

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Intuition behind Singular Value Decomposition

                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                        bull First rotate

                                                                                        bull Second rescale coordinate axes (by )

                                                                                        X

                                                                                        vVSUvVSUvX tt

                                                                                        is

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Intuition behind Singular Value Decomposition

                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                        bull First rotate

                                                                                        bull Second rescale coordinate axes (by )

                                                                                        bull Third rotate again

                                                                                        X

                                                                                        vVSUvVSUvX tt

                                                                                        is

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Intuition behind Singular Value Decomposition

                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                        bull First rotate

                                                                                        bull Second rescale coordinate axes (by )

                                                                                        bull Third rotate again

                                                                                        bull ie have diagonalized the transformation

                                                                                        X

                                                                                        vVSUvVSUvX tt

                                                                                        is

                                                                                        SVD Compact Representation

                                                                                        Useful Labeling

                                                                                        Singular Values in Increasing Order

                                                                                        Review of Linear Algebra (Cont)

                                                                                        )min(1 dnss

                                                                                        SVD Compact Representation

                                                                                        Useful Labeling

                                                                                        Singular Values in Increasing Order

                                                                                        Note singular values = 0 can be omitted

                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        )min(1 dnss

                                                                                        Review of Linear Algebra (Cont)

                                                                                        r

                                                                                        SVD Compact Representation

                                                                                        Useful Labeling

                                                                                        Singular Values in Increasing Order

                                                                                        Note singular values = 0 can be omitted

                                                                                        Let = of positive singular values

                                                                                        )min(1 dnss

                                                                                        Review of Linear Algebra (Cont)

                                                                                        r

                                                                                        SVD Compact Representation

                                                                                        Useful Labeling

                                                                                        Singular Values in Increasing Order

                                                                                        Note singular values = 0 can be omitted

                                                                                        Let = of positive singular values

                                                                                        Then

                                                                                        Where are truncations of

                                                                                        trnrrrd VSUX

                                                                                        VSU

                                                                                        )min(1 dnss

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Full Representation

                                                                                        =

                                                                                        Graphics Display Assumes

                                                                                        ndX ddU ndS nn

                                                                                        tV

                                                                                        nd

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Full Representation

                                                                                        =

                                                                                        Full Rank Basis Matrix

                                                                                        ndX ddU ndS nn

                                                                                        tV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Full Representation

                                                                                        =

                                                                                        Full Rank Basis Matrix

                                                                                        All 0s in Bottom

                                                                                        ndX ddU ndS nn

                                                                                        tV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Reduced Representation

                                                                                        =

                                                                                        These Columns Get 0ed Out

                                                                                        ndX ddU nnS nn

                                                                                        tV

                                                                                        nnd 0

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Reduced Representation

                                                                                        =ndX ndU nnS nn

                                                                                        tV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Reduced Representation

                                                                                        =

                                                                                        Also Some of These May be 0

                                                                                        ndX ndU nnS nn

                                                                                        tV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Compact Representation

                                                                                        = ndX rdU

                                                                                        rrS nrtV

                                                                                        0

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Compact Representation

                                                                                        =

                                                                                        These Get 0ed Out

                                                                                        ndX rdU

                                                                                        rrS nrtV

                                                                                        0

                                                                                        Review of Linear Algebra (Cont)

                                                                                        SVD Compact Representation

                                                                                        = ndX rdU

                                                                                        rrS nrtV

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition

                                                                                        For a (Symmetric) Square Matrix

                                                                                        Find

                                                                                        ddX

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition

                                                                                        For a (Symmetric) Square Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        Called Eigenvalues

                                                                                        Convenient Ordering

                                                                                        ddX

                                                                                        d

                                                                                        D

                                                                                        0

                                                                                        01

                                                                                        n 1

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition

                                                                                        For a (Symmetric) Square Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        And an Orthonormal Matrix

                                                                                        (ie )

                                                                                        ddX

                                                                                        d

                                                                                        D

                                                                                        0

                                                                                        01

                                                                                        ddB

                                                                                        ddtt IBBBB

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition

                                                                                        For a (Symmetric) Square Matrix

                                                                                        Find a Diagonal Matrix

                                                                                        And an Orthonormal Matrix

                                                                                        (ie )

                                                                                        So that ie

                                                                                        ddX

                                                                                        d

                                                                                        D

                                                                                        0

                                                                                        01

                                                                                        ddB

                                                                                        ddtt IBBBB

                                                                                        DBBX tBDBX

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                        (looks similar)

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                        Complex (uses )

                                                                                        VU

                                                                                        1i

                                                                                        Review of Linear Algebra (Cont)

                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                        VU

                                                                                        X

                                                                                        BVU

                                                                                        1i

                                                                                        • Participant Presentations
                                                                                        • PCA to find clusters
                                                                                        • Statistical Smoothing
                                                                                        • Kernel Density Estimation
                                                                                        • Scatterplot Smoothing
                                                                                        • Statistical Smoothing (2)
                                                                                        • SiZer Background
                                                                                        • SiZer Background (2)
                                                                                        • SiZer Background (3)
                                                                                        • SiZer Background (4)
                                                                                        • SiZer Background (5)
                                                                                        • SiZer Overview
                                                                                        • PCA to find clusters (2)
                                                                                        • PCA to find clusters (3)
                                                                                        • PCA to find clusters (4)
                                                                                        • PCA to find clusters (5)
                                                                                        • PCA to find clusters (6)
                                                                                        • PCA to find clusters (7)
                                                                                        • Recall Yeast Cell Cycle Data
                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                        • Frequency 2 Analysis
                                                                                        • Frequency 2 Analysis (2)
                                                                                        • Frequency 2 Analysis (3)
                                                                                        • Yeast Cell Cycle
                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                        • Reclassification of Major Genes
                                                                                        • Compare to Previous Classifrsquon
                                                                                        • New Subpopulation View
                                                                                        • New Subpopulation View (2)
                                                                                        • Detailed Look at PCA
                                                                                        • PCA Rediscovery ndash Renaming
                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                        • An Interesting Historical Note
                                                                                        • An Interesting Historical Note (2)
                                                                                        • Detailed Look at PCA (2)
                                                                                        • Detailed Look at PCA (3)
                                                                                        • Review of Linear Algebra
                                                                                        • Review of Linear Algebra (2)
                                                                                        • Review of Linear Algebra (3)
                                                                                        • Review of Linear Algebra (Cont)
                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                          Review of Linear Algebra

                                                                                          Vector Space

                                                                                          bull set of ldquovectorsrdquo

                                                                                          bull and ldquoscalarsrdquo (coefficients)

                                                                                          bull ldquoclosedrdquo under ldquolinear combinationrdquo

                                                                                          ( in space)

                                                                                          eg

                                                                                          ldquo dim Euclidrsquon spacerdquo

                                                                                          xa

                                                                                          i

                                                                                          ii xa

                                                                                          d

                                                                                          d

                                                                                          d xx

                                                                                          x

                                                                                          x

                                                                                          x 1

                                                                                          1

                                                                                          d

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                          Note Planes not Through the Origin

                                                                                          are not Subspaces

                                                                                          (Do not Contain )00 x

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                          (all Linear Combos of them =

                                                                                          = Containing Hyperplane

                                                                                          through Origin)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Basis of Subspace Set of Vectors that

                                                                                          bull Span ie Everything is a Lin Com of them

                                                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Basis of Subspace Set of Vectors that

                                                                                          bull Span ie Everything is a Lin Com of them

                                                                                          bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                          bull eg ldquoUnit Vector Basisrdquo

                                                                                          bull Since

                                                                                          d

                                                                                          1

                                                                                          0

                                                                                          0

                                                                                          0

                                                                                          1

                                                                                          0

                                                                                          0

                                                                                          0

                                                                                          1

                                                                                          1

                                                                                          0

                                                                                          0

                                                                                          0

                                                                                          1

                                                                                          0

                                                                                          0

                                                                                          0

                                                                                          1

                                                                                          212

                                                                                          1

                                                                                          d

                                                                                          d

                                                                                          xxx

                                                                                          x

                                                                                          x

                                                                                          x

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Basis Matrix of subspace of

                                                                                          Given a basis

                                                                                          create matrix of columns

                                                                                          dnvv 1

                                                                                          nddnd

                                                                                          n

                                                                                          n

                                                                                          vv

                                                                                          vv

                                                                                          vvB

                                                                                          1

                                                                                          111

                                                                                          1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                          where

                                                                                          n

                                                                                          iii aBva

                                                                                          1

                                                                                          na

                                                                                          a

                                                                                          a 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                          where

                                                                                          Note Right Multiplication Gives

                                                                                          Linear Combination of Column Vectors

                                                                                          n

                                                                                          iii aBva

                                                                                          1

                                                                                          na

                                                                                          a

                                                                                          a 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                          where

                                                                                          Check sizes

                                                                                          n

                                                                                          iii aBva

                                                                                          1

                                                                                          na

                                                                                          a

                                                                                          a 1

                                                                                          )1()(1 nndd

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                          For matrices

                                                                                          Define the Matrix Product

                                                                                          mkk

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          nmm

                                                                                          n

                                                                                          bb

                                                                                          bb

                                                                                          B

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iniik

                                                                                          m

                                                                                          iiik

                                                                                          m

                                                                                          inii

                                                                                          m

                                                                                          iii

                                                                                          baba

                                                                                          baba

                                                                                          AB

                                                                                          1

                                                                                          11

                                                                                          11

                                                                                          111

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                          For matrices

                                                                                          Define the Matrix Product

                                                                                          (Inner Products of Rows With Columns )

                                                                                          mkk

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          nmm

                                                                                          n

                                                                                          bb

                                                                                          bb

                                                                                          B

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iniik

                                                                                          m

                                                                                          iiik

                                                                                          m

                                                                                          inii

                                                                                          m

                                                                                          iii

                                                                                          baba

                                                                                          baba

                                                                                          AB

                                                                                          1

                                                                                          11

                                                                                          11

                                                                                          111

                                                                                          A B

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                          For matrices

                                                                                          Define the Matrix Product

                                                                                          (Inner Products of Rows With Columns )

                                                                                          (Composition of Linear Transformations)

                                                                                          mkk

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          nmm

                                                                                          n

                                                                                          bb

                                                                                          bb

                                                                                          B

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iniik

                                                                                          m

                                                                                          iiik

                                                                                          m

                                                                                          inii

                                                                                          m

                                                                                          iii

                                                                                          baba

                                                                                          baba

                                                                                          AB

                                                                                          1

                                                                                          11

                                                                                          11

                                                                                          111

                                                                                          A B

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                          For matrices

                                                                                          Define the Matrix Product

                                                                                          (Inner Products of Rows With Columns )

                                                                                          (Composition of Linear Transformations)

                                                                                          Often Useful to Check Sizes

                                                                                          mkk

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          nmm

                                                                                          n

                                                                                          bb

                                                                                          bb

                                                                                          B

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iniik

                                                                                          m

                                                                                          iiik

                                                                                          m

                                                                                          inii

                                                                                          m

                                                                                          iii

                                                                                          baba

                                                                                          baba

                                                                                          AB

                                                                                          1

                                                                                          11

                                                                                          11

                                                                                          111

                                                                                          nmmknk

                                                                                          A B

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                          For matrices

                                                                                          Define the Matrix Product

                                                                                          (Inner Products of Rows With Columns )

                                                                                          (Composition of Linear Transformations)

                                                                                          Often Useful to Check Sizes

                                                                                          mkk

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          nmm

                                                                                          n

                                                                                          bb

                                                                                          bb

                                                                                          B

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iniik

                                                                                          m

                                                                                          iiik

                                                                                          m

                                                                                          inii

                                                                                          m

                                                                                          iii

                                                                                          baba

                                                                                          baba

                                                                                          AB

                                                                                          1

                                                                                          11

                                                                                          11

                                                                                          111

                                                                                          nmmknk

                                                                                          A B

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Matrix Trace

                                                                                          bull For a Square Matrix

                                                                                          bull Define

                                                                                          mmm

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iiiaAtr

                                                                                          1)(

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Matrix Trace

                                                                                          bull For a Square Matrix

                                                                                          bull Define

                                                                                          bull Trace Commutes with Matrix Multiplication

                                                                                          mmm

                                                                                          m

                                                                                          aa

                                                                                          aa

                                                                                          A

                                                                                          1

                                                                                          111

                                                                                          m

                                                                                          iiiaAtr

                                                                                          1)(

                                                                                          BAtrABtr

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                          bull (Use Basis Above)

                                                                                          bull eg dim of a line is 1

                                                                                          bull eg dim of a plane is 2

                                                                                          dd dim

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                          bull (Use Basis Above)

                                                                                          bull eg dim of a line is 1

                                                                                          bull eg dim of a plane is 2

                                                                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                          (in Statistical Uses eg ANOVA)

                                                                                          dd dim

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Norm of a Vector

                                                                                          bull in d 21

                                                                                          21

                                                                                          1

                                                                                          2 xxxx td

                                                                                          jj

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Norm of a Vector

                                                                                          bull in

                                                                                          bull Idea length of the vector

                                                                                          d 2121

                                                                                          1

                                                                                          2 xxxx td

                                                                                          jj

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Norm of a Vector

                                                                                          bull in

                                                                                          bull Idea length of the vector

                                                                                          bull Note strange properties for high

                                                                                          eg ldquolength of diagonal of unit cuberdquo =

                                                                                          d 2121

                                                                                          1

                                                                                          2 xxxx td

                                                                                          jj

                                                                                          d

                                                                                          d

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Norm of a Vector (cont)

                                                                                          bull Length Normalized Vector

                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                          amp is a Direction Vector)

                                                                                          x

                                                                                          x

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Norm of a Vector (cont)

                                                                                          bull Length Normalized Vector

                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                          amp is a Direction Vector)

                                                                                          bull Define Distance as

                                                                                          x

                                                                                          x

                                                                                          yxyxyxyxd t

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Inner (Dot Scalar) Product

                                                                                          bull for Vectors and

                                                                                          yxyxyx td

                                                                                          jjj

                                                                                          1

                                                                                          x y

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Inner (Dot Scalar) Product

                                                                                          bull for Vectors and

                                                                                          bull Related to Norm via

                                                                                          yxyxyx td

                                                                                          jjj

                                                                                          1

                                                                                          x y

                                                                                          21 xxx

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                          bull measures ldquoangle between and rdquo as

                                                                                          yyxx

                                                                                          yx

                                                                                          yx

                                                                                          yxyxangle

                                                                                          tt

                                                                                          t

                                                                                          11 cos

                                                                                          cos

                                                                                          x y

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                          bull measures ldquoangle between and rdquo as

                                                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                                                          if and only if

                                                                                          yyxx

                                                                                          yx

                                                                                          yx

                                                                                          yxyxangle

                                                                                          tt

                                                                                          t

                                                                                          11 cos

                                                                                          cos

                                                                                          x y

                                                                                          yx 0 yx

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis

                                                                                          bull All Orthogonal to each other

                                                                                          ie for

                                                                                          bull All have Length 1

                                                                                          ie for

                                                                                          nvv 1

                                                                                          1 ii vv

                                                                                          0 ii vv ii

                                                                                          ni 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis (cont)

                                                                                          bull Spectral Representation

                                                                                          where

                                                                                          nvv 1

                                                                                          n

                                                                                          iii vax

                                                                                          1

                                                                                          ii vxa

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis (cont)

                                                                                          bull Spectral Representation

                                                                                          where

                                                                                          (Coefficient is Inner Product Cool Notation)

                                                                                          nvv 1

                                                                                          n

                                                                                          iii vax

                                                                                          1

                                                                                          ii vxa

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis (cont)

                                                                                          bull Spectral Representation

                                                                                          where

                                                                                          Check

                                                                                          nvv 1

                                                                                          n

                                                                                          iii vax

                                                                                          1

                                                                                          ii vxa

                                                                                          iii

                                                                                          n

                                                                                          iii

                                                                                          n

                                                                                          iiii avvavvavx

                                                                                          1

                                                                                          1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis (cont)

                                                                                          bull Spectral Representation

                                                                                          where

                                                                                          Check

                                                                                          bull Matrix Notation where ie

                                                                                          For the Basis Matrix

                                                                                          nvv 1

                                                                                          n

                                                                                          iii vax

                                                                                          1

                                                                                          ii vxa

                                                                                          iii

                                                                                          n

                                                                                          iii

                                                                                          n

                                                                                          iiii avvavvavx

                                                                                          1

                                                                                          1

                                                                                          aBx Bxa tt xBa t

                                                                                          nvvB 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Orthonormal Basis (cont)

                                                                                          bull Spectral Representation

                                                                                          where

                                                                                          Check

                                                                                          bull Matrix Notation where ie

                                                                                          is called transform of

                                                                                          (eg Fourier or Wavelet)

                                                                                          nvv 1

                                                                                          n

                                                                                          iii vax

                                                                                          1

                                                                                          ii vxa

                                                                                          iii

                                                                                          n

                                                                                          iii

                                                                                          n

                                                                                          iiii avvavvavx

                                                                                          1

                                                                                          1

                                                                                          aBx Bxa tt xBa t

                                                                                          xa

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Parseval identity for

                                                                                          in subsp genrsquod by o n basis

                                                                                          x

                                                                                          nvv 1

                                                                                          2

                                                                                          1

                                                                                          22

                                                                                          1

                                                                                          2 aavxx

                                                                                          n

                                                                                          ii

                                                                                          n

                                                                                          ii

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Parseval identity for

                                                                                          in subsp genrsquod by o n basis

                                                                                          bull Pythagorean theorem

                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                          bull ANOVA - sums of squares

                                                                                          x

                                                                                          nvv 1

                                                                                          2

                                                                                          1

                                                                                          22

                                                                                          1

                                                                                          2 aavxx

                                                                                          n

                                                                                          ii

                                                                                          n

                                                                                          ii

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Parseval identity for

                                                                                          in subsp genrsquod by o n basis

                                                                                          bull Pythagorean theorem

                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                          bull ANOVA - sums of squares

                                                                                          bull Transform has same length as

                                                                                          ie ldquorotation in rdquo

                                                                                          x

                                                                                          nvv 1

                                                                                          2

                                                                                          1

                                                                                          22

                                                                                          1

                                                                                          2 aavxx

                                                                                          n

                                                                                          ii

                                                                                          n

                                                                                          ii

                                                                                          a xd

                                                                                          Projection of a Vector onto a Subspace

                                                                                          bull Idea Member of that is Closest to

                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                          Review of Linear Algebra (Cont)x

                                                                                          xV

                                                                                          V

                                                                                          Projection of a Vector onto a Subspace

                                                                                          bull Idea Member of that is Closest to

                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                          bull Find that Solves

                                                                                          (ldquoLeast Squaresrdquo)

                                                                                          Review of Linear Algebra (Cont)x

                                                                                          xV

                                                                                          V

                                                                                          VxPV vxVv

                                                                                          min

                                                                                          Projection of a Vector onto a Subspace

                                                                                          bull Idea Member of that is Closest to

                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                          bull Find that Solves

                                                                                          (ldquoLeast Squaresrdquo)

                                                                                          bull For Inner Product (Hilbert) Space

                                                                                          Exists and is Unique

                                                                                          Review of Linear Algebra (Cont)x

                                                                                          xV

                                                                                          V

                                                                                          VxPV vxVv

                                                                                          min

                                                                                          xPV

                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                          bull General Solution in for Basis Matrix

                                                                                          Review of Linear Algebra (Cont)

                                                                                          d VB

                                                                                          xBBBBxP tVV

                                                                                          tVVV

                                                                                          1

                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                          bull General Solution in for Basis Matrix

                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                          (thus projection is another linear operation)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          d VB

                                                                                          xBBBBxP tVV

                                                                                          tVVV

                                                                                          1

                                                                                          tVV

                                                                                          tVVV BBBBP

                                                                                          1

                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                          bull General Solution in for Basis Matrix

                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                          (thus projection is another linear operation)

                                                                                          (note same operation underlies least squares)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          d VB

                                                                                          xBBBBxP tVV

                                                                                          tVVV

                                                                                          1

                                                                                          tVV

                                                                                          tVVV BBBBP

                                                                                          1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Projection using Orthonormal Basis

                                                                                          bull Basis Matrix is Orthonormal

                                                                                          nnVtV IBB

                                                                                          10

                                                                                          01

                                                                                          1

                                                                                          111

                                                                                          1

                                                                                          1

                                                                                          nnn

                                                                                          n

                                                                                          ntn

                                                                                          t

                                                                                          vvvv

                                                                                          vvvv

                                                                                          vv

                                                                                          v

                                                                                          v

                                                                                          nvv 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Projection using Orthonormal Basis

                                                                                          bull Basis Matrix is Orthonormal

                                                                                          bull So =

                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                          (Recall Right Multrsquon)

                                                                                          nnVtV IBB

                                                                                          10

                                                                                          01

                                                                                          1

                                                                                          111

                                                                                          1

                                                                                          1

                                                                                          nnn

                                                                                          n

                                                                                          ntn

                                                                                          t

                                                                                          vvvv

                                                                                          vvvv

                                                                                          vv

                                                                                          v

                                                                                          v

                                                                                          xBBxP tVVV

                                                                                          x V

                                                                                          nvv 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Projection using Orthonormal Basis (cont)

                                                                                          bull For Orthogonal Complement

                                                                                          and

                                                                                          V

                                                                                          xPxPx VV 222xPxPx VV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Projection using Orthonormal Basis (cont)

                                                                                          bull For Orthogonal Complement

                                                                                          and

                                                                                          bull Parseval Inequality

                                                                                          V

                                                                                          xPxPx VV 222xPxPx VV

                                                                                          2

                                                                                          1

                                                                                          22

                                                                                          1

                                                                                          22 aavxxxP

                                                                                          n

                                                                                          ii

                                                                                          n

                                                                                          iiV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          (Real) Unitary Matrices with

                                                                                          bull Orthonormal Basis Matrix

                                                                                          (So All of Above Applies)

                                                                                          ddU IUU t

                                                                                          Review of Linear Algebra (Cont)

                                                                                          (Real) Unitary Matrices with

                                                                                          bull Orthonormal Basis Matrix

                                                                                          (So All of Above Applies)

                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                          ddU IUU t

                                                                                          yxdyxyxyxUyUxUdn

                                                                                          i ii 2

                                                                                          1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          (Real) Unitary Matrices with

                                                                                          bull Orthonormal Basis Matrix

                                                                                          (So All of Above Applies)

                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                          ddU IUU t

                                                                                          yxdyxyxyxUyUxUdn

                                                                                          i ii 2

                                                                                          1

                                                                                          U

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Singular Value Decomposition (SVD)

                                                                                          For a Matrix

                                                                                          Find

                                                                                          ndX

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Singular Value Decomposition (SVD)

                                                                                          For a Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          with Entries

                                                                                          called Singular Values

                                                                                          ndX

                                                                                          ndS

                                                                                          )min(1 ndss

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Singular Value Decomposition (SVD)

                                                                                          For a Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          with Entries

                                                                                          called Singular Values

                                                                                          And Unitary (Rotation) Matrices

                                                                                          (recall )

                                                                                          ndX

                                                                                          ndS

                                                                                          )min(1 ndss

                                                                                          ddU nnV

                                                                                          IVVUU tt

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Singular Value Decomposition (SVD)

                                                                                          For a Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          with Entries

                                                                                          called Singular Values

                                                                                          And Unitary (Rotation) Matrices

                                                                                          (recall )

                                                                                          So That

                                                                                          ndX

                                                                                          ndS

                                                                                          )min(1 ndss

                                                                                          ddU nnV

                                                                                          IVVUU tt tUSVX

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Intuition behind Singular Value Decomposition

                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                          vVSUvVSUvX tt

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Intuition behind Singular Value Decomposition

                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                          bull First rotate

                                                                                          X

                                                                                          vVSUvVSUvX tt

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Intuition behind Singular Value Decomposition

                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                          bull First rotate

                                                                                          bull Second rescale coordinate axes (by )

                                                                                          X

                                                                                          vVSUvVSUvX tt

                                                                                          is

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Intuition behind Singular Value Decomposition

                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                          bull First rotate

                                                                                          bull Second rescale coordinate axes (by )

                                                                                          bull Third rotate again

                                                                                          X

                                                                                          vVSUvVSUvX tt

                                                                                          is

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Intuition behind Singular Value Decomposition

                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                          bull First rotate

                                                                                          bull Second rescale coordinate axes (by )

                                                                                          bull Third rotate again

                                                                                          bull ie have diagonalized the transformation

                                                                                          X

                                                                                          vVSUvVSUvX tt

                                                                                          is

                                                                                          SVD Compact Representation

                                                                                          Useful Labeling

                                                                                          Singular Values in Increasing Order

                                                                                          Review of Linear Algebra (Cont)

                                                                                          )min(1 dnss

                                                                                          SVD Compact Representation

                                                                                          Useful Labeling

                                                                                          Singular Values in Increasing Order

                                                                                          Note singular values = 0 can be omitted

                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          )min(1 dnss

                                                                                          Review of Linear Algebra (Cont)

                                                                                          r

                                                                                          SVD Compact Representation

                                                                                          Useful Labeling

                                                                                          Singular Values in Increasing Order

                                                                                          Note singular values = 0 can be omitted

                                                                                          Let = of positive singular values

                                                                                          )min(1 dnss

                                                                                          Review of Linear Algebra (Cont)

                                                                                          r

                                                                                          SVD Compact Representation

                                                                                          Useful Labeling

                                                                                          Singular Values in Increasing Order

                                                                                          Note singular values = 0 can be omitted

                                                                                          Let = of positive singular values

                                                                                          Then

                                                                                          Where are truncations of

                                                                                          trnrrrd VSUX

                                                                                          VSU

                                                                                          )min(1 dnss

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Full Representation

                                                                                          =

                                                                                          Graphics Display Assumes

                                                                                          ndX ddU ndS nn

                                                                                          tV

                                                                                          nd

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Full Representation

                                                                                          =

                                                                                          Full Rank Basis Matrix

                                                                                          ndX ddU ndS nn

                                                                                          tV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Full Representation

                                                                                          =

                                                                                          Full Rank Basis Matrix

                                                                                          All 0s in Bottom

                                                                                          ndX ddU ndS nn

                                                                                          tV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Reduced Representation

                                                                                          =

                                                                                          These Columns Get 0ed Out

                                                                                          ndX ddU nnS nn

                                                                                          tV

                                                                                          nnd 0

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Reduced Representation

                                                                                          =ndX ndU nnS nn

                                                                                          tV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Reduced Representation

                                                                                          =

                                                                                          Also Some of These May be 0

                                                                                          ndX ndU nnS nn

                                                                                          tV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Compact Representation

                                                                                          = ndX rdU

                                                                                          rrS nrtV

                                                                                          0

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Compact Representation

                                                                                          =

                                                                                          These Get 0ed Out

                                                                                          ndX rdU

                                                                                          rrS nrtV

                                                                                          0

                                                                                          Review of Linear Algebra (Cont)

                                                                                          SVD Compact Representation

                                                                                          = ndX rdU

                                                                                          rrS nrtV

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition

                                                                                          For a (Symmetric) Square Matrix

                                                                                          Find

                                                                                          ddX

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition

                                                                                          For a (Symmetric) Square Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          Called Eigenvalues

                                                                                          Convenient Ordering

                                                                                          ddX

                                                                                          d

                                                                                          D

                                                                                          0

                                                                                          01

                                                                                          n 1

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition

                                                                                          For a (Symmetric) Square Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          And an Orthonormal Matrix

                                                                                          (ie )

                                                                                          ddX

                                                                                          d

                                                                                          D

                                                                                          0

                                                                                          01

                                                                                          ddB

                                                                                          ddtt IBBBB

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition

                                                                                          For a (Symmetric) Square Matrix

                                                                                          Find a Diagonal Matrix

                                                                                          And an Orthonormal Matrix

                                                                                          (ie )

                                                                                          So that ie

                                                                                          ddX

                                                                                          d

                                                                                          D

                                                                                          0

                                                                                          01

                                                                                          ddB

                                                                                          ddtt IBBBB

                                                                                          DBBX tBDBX

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                          (looks similar)

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                          Complex (uses )

                                                                                          VU

                                                                                          1i

                                                                                          Review of Linear Algebra (Cont)

                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                          VU

                                                                                          X

                                                                                          BVU

                                                                                          1i

                                                                                          • Participant Presentations
                                                                                          • PCA to find clusters
                                                                                          • Statistical Smoothing
                                                                                          • Kernel Density Estimation
                                                                                          • Scatterplot Smoothing
                                                                                          • Statistical Smoothing (2)
                                                                                          • SiZer Background
                                                                                          • SiZer Background (2)
                                                                                          • SiZer Background (3)
                                                                                          • SiZer Background (4)
                                                                                          • SiZer Background (5)
                                                                                          • SiZer Overview
                                                                                          • PCA to find clusters (2)
                                                                                          • PCA to find clusters (3)
                                                                                          • PCA to find clusters (4)
                                                                                          • PCA to find clusters (5)
                                                                                          • PCA to find clusters (6)
                                                                                          • PCA to find clusters (7)
                                                                                          • Recall Yeast Cell Cycle Data
                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                          • Frequency 2 Analysis
                                                                                          • Frequency 2 Analysis (2)
                                                                                          • Frequency 2 Analysis (3)
                                                                                          • Yeast Cell Cycle
                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                          • Reclassification of Major Genes
                                                                                          • Compare to Previous Classifrsquon
                                                                                          • New Subpopulation View
                                                                                          • New Subpopulation View (2)
                                                                                          • Detailed Look at PCA
                                                                                          • PCA Rediscovery ndash Renaming
                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                          • An Interesting Historical Note
                                                                                          • An Interesting Historical Note (2)
                                                                                          • Detailed Look at PCA (2)
                                                                                          • Detailed Look at PCA (3)
                                                                                          • Review of Linear Algebra
                                                                                          • Review of Linear Algebra (2)
                                                                                          • Review of Linear Algebra (3)
                                                                                          • Review of Linear Algebra (Cont)
                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combination

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                            Note Planes not Through the Origin

                                                                                            are not Subspaces

                                                                                            (Do not Contain )00 x

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                            (all Linear Combos of them =

                                                                                            = Containing Hyperplane

                                                                                            through Origin)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Basis of Subspace Set of Vectors that

                                                                                            bull Span ie Everything is a Lin Com of them

                                                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Basis of Subspace Set of Vectors that

                                                                                            bull Span ie Everything is a Lin Com of them

                                                                                            bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                            bull eg ldquoUnit Vector Basisrdquo

                                                                                            bull Since

                                                                                            d

                                                                                            1

                                                                                            0

                                                                                            0

                                                                                            0

                                                                                            1

                                                                                            0

                                                                                            0

                                                                                            0

                                                                                            1

                                                                                            1

                                                                                            0

                                                                                            0

                                                                                            0

                                                                                            1

                                                                                            0

                                                                                            0

                                                                                            0

                                                                                            1

                                                                                            212

                                                                                            1

                                                                                            d

                                                                                            d

                                                                                            xxx

                                                                                            x

                                                                                            x

                                                                                            x

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Basis Matrix of subspace of

                                                                                            Given a basis

                                                                                            create matrix of columns

                                                                                            dnvv 1

                                                                                            nddnd

                                                                                            n

                                                                                            n

                                                                                            vv

                                                                                            vv

                                                                                            vvB

                                                                                            1

                                                                                            111

                                                                                            1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                                            where

                                                                                            n

                                                                                            iii aBva

                                                                                            1

                                                                                            na

                                                                                            a

                                                                                            a 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                                            where

                                                                                            Note Right Multiplication Gives

                                                                                            Linear Combination of Column Vectors

                                                                                            n

                                                                                            iii aBva

                                                                                            1

                                                                                            na

                                                                                            a

                                                                                            a 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                                            where

                                                                                            Check sizes

                                                                                            n

                                                                                            iii aBva

                                                                                            1

                                                                                            na

                                                                                            a

                                                                                            a 1

                                                                                            )1()(1 nndd

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                            For matrices

                                                                                            Define the Matrix Product

                                                                                            mkk

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            nmm

                                                                                            n

                                                                                            bb

                                                                                            bb

                                                                                            B

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iniik

                                                                                            m

                                                                                            iiik

                                                                                            m

                                                                                            inii

                                                                                            m

                                                                                            iii

                                                                                            baba

                                                                                            baba

                                                                                            AB

                                                                                            1

                                                                                            11

                                                                                            11

                                                                                            111

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                            For matrices

                                                                                            Define the Matrix Product

                                                                                            (Inner Products of Rows With Columns )

                                                                                            mkk

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            nmm

                                                                                            n

                                                                                            bb

                                                                                            bb

                                                                                            B

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iniik

                                                                                            m

                                                                                            iiik

                                                                                            m

                                                                                            inii

                                                                                            m

                                                                                            iii

                                                                                            baba

                                                                                            baba

                                                                                            AB

                                                                                            1

                                                                                            11

                                                                                            11

                                                                                            111

                                                                                            A B

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                            For matrices

                                                                                            Define the Matrix Product

                                                                                            (Inner Products of Rows With Columns )

                                                                                            (Composition of Linear Transformations)

                                                                                            mkk

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            nmm

                                                                                            n

                                                                                            bb

                                                                                            bb

                                                                                            B

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iniik

                                                                                            m

                                                                                            iiik

                                                                                            m

                                                                                            inii

                                                                                            m

                                                                                            iii

                                                                                            baba

                                                                                            baba

                                                                                            AB

                                                                                            1

                                                                                            11

                                                                                            11

                                                                                            111

                                                                                            A B

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                            For matrices

                                                                                            Define the Matrix Product

                                                                                            (Inner Products of Rows With Columns )

                                                                                            (Composition of Linear Transformations)

                                                                                            Often Useful to Check Sizes

                                                                                            mkk

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            nmm

                                                                                            n

                                                                                            bb

                                                                                            bb

                                                                                            B

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iniik

                                                                                            m

                                                                                            iiik

                                                                                            m

                                                                                            inii

                                                                                            m

                                                                                            iii

                                                                                            baba

                                                                                            baba

                                                                                            AB

                                                                                            1

                                                                                            11

                                                                                            11

                                                                                            111

                                                                                            nmmknk

                                                                                            A B

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                            For matrices

                                                                                            Define the Matrix Product

                                                                                            (Inner Products of Rows With Columns )

                                                                                            (Composition of Linear Transformations)

                                                                                            Often Useful to Check Sizes

                                                                                            mkk

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            nmm

                                                                                            n

                                                                                            bb

                                                                                            bb

                                                                                            B

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iniik

                                                                                            m

                                                                                            iiik

                                                                                            m

                                                                                            inii

                                                                                            m

                                                                                            iii

                                                                                            baba

                                                                                            baba

                                                                                            AB

                                                                                            1

                                                                                            11

                                                                                            11

                                                                                            111

                                                                                            nmmknk

                                                                                            A B

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Matrix Trace

                                                                                            bull For a Square Matrix

                                                                                            bull Define

                                                                                            mmm

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iiiaAtr

                                                                                            1)(

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Matrix Trace

                                                                                            bull For a Square Matrix

                                                                                            bull Define

                                                                                            bull Trace Commutes with Matrix Multiplication

                                                                                            mmm

                                                                                            m

                                                                                            aa

                                                                                            aa

                                                                                            A

                                                                                            1

                                                                                            111

                                                                                            m

                                                                                            iiiaAtr

                                                                                            1)(

                                                                                            BAtrABtr

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                            bull (Use Basis Above)

                                                                                            bull eg dim of a line is 1

                                                                                            bull eg dim of a plane is 2

                                                                                            dd dim

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                            bull (Use Basis Above)

                                                                                            bull eg dim of a line is 1

                                                                                            bull eg dim of a plane is 2

                                                                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                            (in Statistical Uses eg ANOVA)

                                                                                            dd dim

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Norm of a Vector

                                                                                            bull in d 21

                                                                                            21

                                                                                            1

                                                                                            2 xxxx td

                                                                                            jj

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Norm of a Vector

                                                                                            bull in

                                                                                            bull Idea length of the vector

                                                                                            d 2121

                                                                                            1

                                                                                            2 xxxx td

                                                                                            jj

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Norm of a Vector

                                                                                            bull in

                                                                                            bull Idea length of the vector

                                                                                            bull Note strange properties for high

                                                                                            eg ldquolength of diagonal of unit cuberdquo =

                                                                                            d 2121

                                                                                            1

                                                                                            2 xxxx td

                                                                                            jj

                                                                                            d

                                                                                            d

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Norm of a Vector (cont)

                                                                                            bull Length Normalized Vector

                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                            amp is a Direction Vector)

                                                                                            x

                                                                                            x

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Norm of a Vector (cont)

                                                                                            bull Length Normalized Vector

                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                            amp is a Direction Vector)

                                                                                            bull Define Distance as

                                                                                            x

                                                                                            x

                                                                                            yxyxyxyxd t

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Inner (Dot Scalar) Product

                                                                                            bull for Vectors and

                                                                                            yxyxyx td

                                                                                            jjj

                                                                                            1

                                                                                            x y

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Inner (Dot Scalar) Product

                                                                                            bull for Vectors and

                                                                                            bull Related to Norm via

                                                                                            yxyxyx td

                                                                                            jjj

                                                                                            1

                                                                                            x y

                                                                                            21 xxx

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                            bull measures ldquoangle between and rdquo as

                                                                                            yyxx

                                                                                            yx

                                                                                            yx

                                                                                            yxyxangle

                                                                                            tt

                                                                                            t

                                                                                            11 cos

                                                                                            cos

                                                                                            x y

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                            bull measures ldquoangle between and rdquo as

                                                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                                                            if and only if

                                                                                            yyxx

                                                                                            yx

                                                                                            yx

                                                                                            yxyxangle

                                                                                            tt

                                                                                            t

                                                                                            11 cos

                                                                                            cos

                                                                                            x y

                                                                                            yx 0 yx

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis

                                                                                            bull All Orthogonal to each other

                                                                                            ie for

                                                                                            bull All have Length 1

                                                                                            ie for

                                                                                            nvv 1

                                                                                            1 ii vv

                                                                                            0 ii vv ii

                                                                                            ni 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis (cont)

                                                                                            bull Spectral Representation

                                                                                            where

                                                                                            nvv 1

                                                                                            n

                                                                                            iii vax

                                                                                            1

                                                                                            ii vxa

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis (cont)

                                                                                            bull Spectral Representation

                                                                                            where

                                                                                            (Coefficient is Inner Product Cool Notation)

                                                                                            nvv 1

                                                                                            n

                                                                                            iii vax

                                                                                            1

                                                                                            ii vxa

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis (cont)

                                                                                            bull Spectral Representation

                                                                                            where

                                                                                            Check

                                                                                            nvv 1

                                                                                            n

                                                                                            iii vax

                                                                                            1

                                                                                            ii vxa

                                                                                            iii

                                                                                            n

                                                                                            iii

                                                                                            n

                                                                                            iiii avvavvavx

                                                                                            1

                                                                                            1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis (cont)

                                                                                            bull Spectral Representation

                                                                                            where

                                                                                            Check

                                                                                            bull Matrix Notation where ie

                                                                                            For the Basis Matrix

                                                                                            nvv 1

                                                                                            n

                                                                                            iii vax

                                                                                            1

                                                                                            ii vxa

                                                                                            iii

                                                                                            n

                                                                                            iii

                                                                                            n

                                                                                            iiii avvavvavx

                                                                                            1

                                                                                            1

                                                                                            aBx Bxa tt xBa t

                                                                                            nvvB 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Orthonormal Basis (cont)

                                                                                            bull Spectral Representation

                                                                                            where

                                                                                            Check

                                                                                            bull Matrix Notation where ie

                                                                                            is called transform of

                                                                                            (eg Fourier or Wavelet)

                                                                                            nvv 1

                                                                                            n

                                                                                            iii vax

                                                                                            1

                                                                                            ii vxa

                                                                                            iii

                                                                                            n

                                                                                            iii

                                                                                            n

                                                                                            iiii avvavvavx

                                                                                            1

                                                                                            1

                                                                                            aBx Bxa tt xBa t

                                                                                            xa

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Parseval identity for

                                                                                            in subsp genrsquod by o n basis

                                                                                            x

                                                                                            nvv 1

                                                                                            2

                                                                                            1

                                                                                            22

                                                                                            1

                                                                                            2 aavxx

                                                                                            n

                                                                                            ii

                                                                                            n

                                                                                            ii

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Parseval identity for

                                                                                            in subsp genrsquod by o n basis

                                                                                            bull Pythagorean theorem

                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                            bull ANOVA - sums of squares

                                                                                            x

                                                                                            nvv 1

                                                                                            2

                                                                                            1

                                                                                            22

                                                                                            1

                                                                                            2 aavxx

                                                                                            n

                                                                                            ii

                                                                                            n

                                                                                            ii

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Parseval identity for

                                                                                            in subsp genrsquod by o n basis

                                                                                            bull Pythagorean theorem

                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                            bull ANOVA - sums of squares

                                                                                            bull Transform has same length as

                                                                                            ie ldquorotation in rdquo

                                                                                            x

                                                                                            nvv 1

                                                                                            2

                                                                                            1

                                                                                            22

                                                                                            1

                                                                                            2 aavxx

                                                                                            n

                                                                                            ii

                                                                                            n

                                                                                            ii

                                                                                            a xd

                                                                                            Projection of a Vector onto a Subspace

                                                                                            bull Idea Member of that is Closest to

                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                            Review of Linear Algebra (Cont)x

                                                                                            xV

                                                                                            V

                                                                                            Projection of a Vector onto a Subspace

                                                                                            bull Idea Member of that is Closest to

                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                            bull Find that Solves

                                                                                            (ldquoLeast Squaresrdquo)

                                                                                            Review of Linear Algebra (Cont)x

                                                                                            xV

                                                                                            V

                                                                                            VxPV vxVv

                                                                                            min

                                                                                            Projection of a Vector onto a Subspace

                                                                                            bull Idea Member of that is Closest to

                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                            bull Find that Solves

                                                                                            (ldquoLeast Squaresrdquo)

                                                                                            bull For Inner Product (Hilbert) Space

                                                                                            Exists and is Unique

                                                                                            Review of Linear Algebra (Cont)x

                                                                                            xV

                                                                                            V

                                                                                            VxPV vxVv

                                                                                            min

                                                                                            xPV

                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                            bull General Solution in for Basis Matrix

                                                                                            Review of Linear Algebra (Cont)

                                                                                            d VB

                                                                                            xBBBBxP tVV

                                                                                            tVVV

                                                                                            1

                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                            bull General Solution in for Basis Matrix

                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                            (thus projection is another linear operation)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            d VB

                                                                                            xBBBBxP tVV

                                                                                            tVVV

                                                                                            1

                                                                                            tVV

                                                                                            tVVV BBBBP

                                                                                            1

                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                            bull General Solution in for Basis Matrix

                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                            (thus projection is another linear operation)

                                                                                            (note same operation underlies least squares)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            d VB

                                                                                            xBBBBxP tVV

                                                                                            tVVV

                                                                                            1

                                                                                            tVV

                                                                                            tVVV BBBBP

                                                                                            1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Projection using Orthonormal Basis

                                                                                            bull Basis Matrix is Orthonormal

                                                                                            nnVtV IBB

                                                                                            10

                                                                                            01

                                                                                            1

                                                                                            111

                                                                                            1

                                                                                            1

                                                                                            nnn

                                                                                            n

                                                                                            ntn

                                                                                            t

                                                                                            vvvv

                                                                                            vvvv

                                                                                            vv

                                                                                            v

                                                                                            v

                                                                                            nvv 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Projection using Orthonormal Basis

                                                                                            bull Basis Matrix is Orthonormal

                                                                                            bull So =

                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                            (Recall Right Multrsquon)

                                                                                            nnVtV IBB

                                                                                            10

                                                                                            01

                                                                                            1

                                                                                            111

                                                                                            1

                                                                                            1

                                                                                            nnn

                                                                                            n

                                                                                            ntn

                                                                                            t

                                                                                            vvvv

                                                                                            vvvv

                                                                                            vv

                                                                                            v

                                                                                            v

                                                                                            xBBxP tVVV

                                                                                            x V

                                                                                            nvv 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Projection using Orthonormal Basis (cont)

                                                                                            bull For Orthogonal Complement

                                                                                            and

                                                                                            V

                                                                                            xPxPx VV 222xPxPx VV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Projection using Orthonormal Basis (cont)

                                                                                            bull For Orthogonal Complement

                                                                                            and

                                                                                            bull Parseval Inequality

                                                                                            V

                                                                                            xPxPx VV 222xPxPx VV

                                                                                            2

                                                                                            1

                                                                                            22

                                                                                            1

                                                                                            22 aavxxxP

                                                                                            n

                                                                                            ii

                                                                                            n

                                                                                            iiV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            (Real) Unitary Matrices with

                                                                                            bull Orthonormal Basis Matrix

                                                                                            (So All of Above Applies)

                                                                                            ddU IUU t

                                                                                            Review of Linear Algebra (Cont)

                                                                                            (Real) Unitary Matrices with

                                                                                            bull Orthonormal Basis Matrix

                                                                                            (So All of Above Applies)

                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                            ddU IUU t

                                                                                            yxdyxyxyxUyUxUdn

                                                                                            i ii 2

                                                                                            1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            (Real) Unitary Matrices with

                                                                                            bull Orthonormal Basis Matrix

                                                                                            (So All of Above Applies)

                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                            ddU IUU t

                                                                                            yxdyxyxyxUyUxUdn

                                                                                            i ii 2

                                                                                            1

                                                                                            U

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Singular Value Decomposition (SVD)

                                                                                            For a Matrix

                                                                                            Find

                                                                                            ndX

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Singular Value Decomposition (SVD)

                                                                                            For a Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            with Entries

                                                                                            called Singular Values

                                                                                            ndX

                                                                                            ndS

                                                                                            )min(1 ndss

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Singular Value Decomposition (SVD)

                                                                                            For a Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            with Entries

                                                                                            called Singular Values

                                                                                            And Unitary (Rotation) Matrices

                                                                                            (recall )

                                                                                            ndX

                                                                                            ndS

                                                                                            )min(1 ndss

                                                                                            ddU nnV

                                                                                            IVVUU tt

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Singular Value Decomposition (SVD)

                                                                                            For a Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            with Entries

                                                                                            called Singular Values

                                                                                            And Unitary (Rotation) Matrices

                                                                                            (recall )

                                                                                            So That

                                                                                            ndX

                                                                                            ndS

                                                                                            )min(1 ndss

                                                                                            ddU nnV

                                                                                            IVVUU tt tUSVX

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Intuition behind Singular Value Decomposition

                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                            vVSUvVSUvX tt

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Intuition behind Singular Value Decomposition

                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                            bull First rotate

                                                                                            X

                                                                                            vVSUvVSUvX tt

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Intuition behind Singular Value Decomposition

                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                            bull First rotate

                                                                                            bull Second rescale coordinate axes (by )

                                                                                            X

                                                                                            vVSUvVSUvX tt

                                                                                            is

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Intuition behind Singular Value Decomposition

                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                            bull First rotate

                                                                                            bull Second rescale coordinate axes (by )

                                                                                            bull Third rotate again

                                                                                            X

                                                                                            vVSUvVSUvX tt

                                                                                            is

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Intuition behind Singular Value Decomposition

                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                            bull First rotate

                                                                                            bull Second rescale coordinate axes (by )

                                                                                            bull Third rotate again

                                                                                            bull ie have diagonalized the transformation

                                                                                            X

                                                                                            vVSUvVSUvX tt

                                                                                            is

                                                                                            SVD Compact Representation

                                                                                            Useful Labeling

                                                                                            Singular Values in Increasing Order

                                                                                            Review of Linear Algebra (Cont)

                                                                                            )min(1 dnss

                                                                                            SVD Compact Representation

                                                                                            Useful Labeling

                                                                                            Singular Values in Increasing Order

                                                                                            Note singular values = 0 can be omitted

                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            )min(1 dnss

                                                                                            Review of Linear Algebra (Cont)

                                                                                            r

                                                                                            SVD Compact Representation

                                                                                            Useful Labeling

                                                                                            Singular Values in Increasing Order

                                                                                            Note singular values = 0 can be omitted

                                                                                            Let = of positive singular values

                                                                                            )min(1 dnss

                                                                                            Review of Linear Algebra (Cont)

                                                                                            r

                                                                                            SVD Compact Representation

                                                                                            Useful Labeling

                                                                                            Singular Values in Increasing Order

                                                                                            Note singular values = 0 can be omitted

                                                                                            Let = of positive singular values

                                                                                            Then

                                                                                            Where are truncations of

                                                                                            trnrrrd VSUX

                                                                                            VSU

                                                                                            )min(1 dnss

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Full Representation

                                                                                            =

                                                                                            Graphics Display Assumes

                                                                                            ndX ddU ndS nn

                                                                                            tV

                                                                                            nd

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Full Representation

                                                                                            =

                                                                                            Full Rank Basis Matrix

                                                                                            ndX ddU ndS nn

                                                                                            tV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Full Representation

                                                                                            =

                                                                                            Full Rank Basis Matrix

                                                                                            All 0s in Bottom

                                                                                            ndX ddU ndS nn

                                                                                            tV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Reduced Representation

                                                                                            =

                                                                                            These Columns Get 0ed Out

                                                                                            ndX ddU nnS nn

                                                                                            tV

                                                                                            nnd 0

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Reduced Representation

                                                                                            =ndX ndU nnS nn

                                                                                            tV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Reduced Representation

                                                                                            =

                                                                                            Also Some of These May be 0

                                                                                            ndX ndU nnS nn

                                                                                            tV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Compact Representation

                                                                                            = ndX rdU

                                                                                            rrS nrtV

                                                                                            0

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Compact Representation

                                                                                            =

                                                                                            These Get 0ed Out

                                                                                            ndX rdU

                                                                                            rrS nrtV

                                                                                            0

                                                                                            Review of Linear Algebra (Cont)

                                                                                            SVD Compact Representation

                                                                                            = ndX rdU

                                                                                            rrS nrtV

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition

                                                                                            For a (Symmetric) Square Matrix

                                                                                            Find

                                                                                            ddX

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition

                                                                                            For a (Symmetric) Square Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            Called Eigenvalues

                                                                                            Convenient Ordering

                                                                                            ddX

                                                                                            d

                                                                                            D

                                                                                            0

                                                                                            01

                                                                                            n 1

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition

                                                                                            For a (Symmetric) Square Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            And an Orthonormal Matrix

                                                                                            (ie )

                                                                                            ddX

                                                                                            d

                                                                                            D

                                                                                            0

                                                                                            01

                                                                                            ddB

                                                                                            ddtt IBBBB

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition

                                                                                            For a (Symmetric) Square Matrix

                                                                                            Find a Diagonal Matrix

                                                                                            And an Orthonormal Matrix

                                                                                            (ie )

                                                                                            So that ie

                                                                                            ddX

                                                                                            d

                                                                                            D

                                                                                            0

                                                                                            01

                                                                                            ddB

                                                                                            ddtt IBBBB

                                                                                            DBBX tBDBX

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                            (looks similar)

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                            Complex (uses )

                                                                                            VU

                                                                                            1i

                                                                                            Review of Linear Algebra (Cont)

                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                            VU

                                                                                            X

                                                                                            BVU

                                                                                            1i

                                                                                            • Participant Presentations
                                                                                            • PCA to find clusters
                                                                                            • Statistical Smoothing
                                                                                            • Kernel Density Estimation
                                                                                            • Scatterplot Smoothing
                                                                                            • Statistical Smoothing (2)
                                                                                            • SiZer Background
                                                                                            • SiZer Background (2)
                                                                                            • SiZer Background (3)
                                                                                            • SiZer Background (4)
                                                                                            • SiZer Background (5)
                                                                                            • SiZer Overview
                                                                                            • PCA to find clusters (2)
                                                                                            • PCA to find clusters (3)
                                                                                            • PCA to find clusters (4)
                                                                                            • PCA to find clusters (5)
                                                                                            • PCA to find clusters (6)
                                                                                            • PCA to find clusters (7)
                                                                                            • Recall Yeast Cell Cycle Data
                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                            • Frequency 2 Analysis
                                                                                            • Frequency 2 Analysis (2)
                                                                                            • Frequency 2 Analysis (3)
                                                                                            • Yeast Cell Cycle
                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                            • Reclassification of Major Genes
                                                                                            • Compare to Previous Classifrsquon
                                                                                            • New Subpopulation View
                                                                                            • New Subpopulation View (2)
                                                                                            • Detailed Look at PCA
                                                                                            • PCA Rediscovery ndash Renaming
                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                            • An Interesting Historical Note
                                                                                            • An Interesting Historical Note (2)
                                                                                            • Detailed Look at PCA (2)
                                                                                            • Detailed Look at PCA (3)
                                                                                            • Review of Linear Algebra
                                                                                            • Review of Linear Algebra (2)
                                                                                            • Review of Linear Algebra (3)
                                                                                            • Review of Linear Algebra (Cont)
                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                              Note Planes not Through the Origin

                                                                                              are not Subspaces

                                                                                              (Do not Contain )00 x

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                              (all Linear Combos of them =

                                                                                              = Containing Hyperplane

                                                                                              through Origin)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Basis of Subspace Set of Vectors that

                                                                                              bull Span ie Everything is a Lin Com of them

                                                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Basis of Subspace Set of Vectors that

                                                                                              bull Span ie Everything is a Lin Com of them

                                                                                              bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                              bull eg ldquoUnit Vector Basisrdquo

                                                                                              bull Since

                                                                                              d

                                                                                              1

                                                                                              0

                                                                                              0

                                                                                              0

                                                                                              1

                                                                                              0

                                                                                              0

                                                                                              0

                                                                                              1

                                                                                              1

                                                                                              0

                                                                                              0

                                                                                              0

                                                                                              1

                                                                                              0

                                                                                              0

                                                                                              0

                                                                                              1

                                                                                              212

                                                                                              1

                                                                                              d

                                                                                              d

                                                                                              xxx

                                                                                              x

                                                                                              x

                                                                                              x

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Basis Matrix of subspace of

                                                                                              Given a basis

                                                                                              create matrix of columns

                                                                                              dnvv 1

                                                                                              nddnd

                                                                                              n

                                                                                              n

                                                                                              vv

                                                                                              vv

                                                                                              vvB

                                                                                              1

                                                                                              111

                                                                                              1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                                              where

                                                                                              n

                                                                                              iii aBva

                                                                                              1

                                                                                              na

                                                                                              a

                                                                                              a 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                                              where

                                                                                              Note Right Multiplication Gives

                                                                                              Linear Combination of Column Vectors

                                                                                              n

                                                                                              iii aBva

                                                                                              1

                                                                                              na

                                                                                              a

                                                                                              a 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                                              where

                                                                                              Check sizes

                                                                                              n

                                                                                              iii aBva

                                                                                              1

                                                                                              na

                                                                                              a

                                                                                              a 1

                                                                                              )1()(1 nndd

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                              For matrices

                                                                                              Define the Matrix Product

                                                                                              mkk

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              nmm

                                                                                              n

                                                                                              bb

                                                                                              bb

                                                                                              B

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iniik

                                                                                              m

                                                                                              iiik

                                                                                              m

                                                                                              inii

                                                                                              m

                                                                                              iii

                                                                                              baba

                                                                                              baba

                                                                                              AB

                                                                                              1

                                                                                              11

                                                                                              11

                                                                                              111

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                              For matrices

                                                                                              Define the Matrix Product

                                                                                              (Inner Products of Rows With Columns )

                                                                                              mkk

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              nmm

                                                                                              n

                                                                                              bb

                                                                                              bb

                                                                                              B

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iniik

                                                                                              m

                                                                                              iiik

                                                                                              m

                                                                                              inii

                                                                                              m

                                                                                              iii

                                                                                              baba

                                                                                              baba

                                                                                              AB

                                                                                              1

                                                                                              11

                                                                                              11

                                                                                              111

                                                                                              A B

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                              For matrices

                                                                                              Define the Matrix Product

                                                                                              (Inner Products of Rows With Columns )

                                                                                              (Composition of Linear Transformations)

                                                                                              mkk

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              nmm

                                                                                              n

                                                                                              bb

                                                                                              bb

                                                                                              B

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iniik

                                                                                              m

                                                                                              iiik

                                                                                              m

                                                                                              inii

                                                                                              m

                                                                                              iii

                                                                                              baba

                                                                                              baba

                                                                                              AB

                                                                                              1

                                                                                              11

                                                                                              11

                                                                                              111

                                                                                              A B

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                              For matrices

                                                                                              Define the Matrix Product

                                                                                              (Inner Products of Rows With Columns )

                                                                                              (Composition of Linear Transformations)

                                                                                              Often Useful to Check Sizes

                                                                                              mkk

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              nmm

                                                                                              n

                                                                                              bb

                                                                                              bb

                                                                                              B

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iniik

                                                                                              m

                                                                                              iiik

                                                                                              m

                                                                                              inii

                                                                                              m

                                                                                              iii

                                                                                              baba

                                                                                              baba

                                                                                              AB

                                                                                              1

                                                                                              11

                                                                                              11

                                                                                              111

                                                                                              nmmknk

                                                                                              A B

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                              For matrices

                                                                                              Define the Matrix Product

                                                                                              (Inner Products of Rows With Columns )

                                                                                              (Composition of Linear Transformations)

                                                                                              Often Useful to Check Sizes

                                                                                              mkk

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              nmm

                                                                                              n

                                                                                              bb

                                                                                              bb

                                                                                              B

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iniik

                                                                                              m

                                                                                              iiik

                                                                                              m

                                                                                              inii

                                                                                              m

                                                                                              iii

                                                                                              baba

                                                                                              baba

                                                                                              AB

                                                                                              1

                                                                                              11

                                                                                              11

                                                                                              111

                                                                                              nmmknk

                                                                                              A B

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Matrix Trace

                                                                                              bull For a Square Matrix

                                                                                              bull Define

                                                                                              mmm

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iiiaAtr

                                                                                              1)(

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Matrix Trace

                                                                                              bull For a Square Matrix

                                                                                              bull Define

                                                                                              bull Trace Commutes with Matrix Multiplication

                                                                                              mmm

                                                                                              m

                                                                                              aa

                                                                                              aa

                                                                                              A

                                                                                              1

                                                                                              111

                                                                                              m

                                                                                              iiiaAtr

                                                                                              1)(

                                                                                              BAtrABtr

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                              bull (Use Basis Above)

                                                                                              bull eg dim of a line is 1

                                                                                              bull eg dim of a plane is 2

                                                                                              dd dim

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                              bull (Use Basis Above)

                                                                                              bull eg dim of a line is 1

                                                                                              bull eg dim of a plane is 2

                                                                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                              (in Statistical Uses eg ANOVA)

                                                                                              dd dim

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Norm of a Vector

                                                                                              bull in d 21

                                                                                              21

                                                                                              1

                                                                                              2 xxxx td

                                                                                              jj

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Norm of a Vector

                                                                                              bull in

                                                                                              bull Idea length of the vector

                                                                                              d 2121

                                                                                              1

                                                                                              2 xxxx td

                                                                                              jj

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Norm of a Vector

                                                                                              bull in

                                                                                              bull Idea length of the vector

                                                                                              bull Note strange properties for high

                                                                                              eg ldquolength of diagonal of unit cuberdquo =

                                                                                              d 2121

                                                                                              1

                                                                                              2 xxxx td

                                                                                              jj

                                                                                              d

                                                                                              d

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Norm of a Vector (cont)

                                                                                              bull Length Normalized Vector

                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                              amp is a Direction Vector)

                                                                                              x

                                                                                              x

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Norm of a Vector (cont)

                                                                                              bull Length Normalized Vector

                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                              amp is a Direction Vector)

                                                                                              bull Define Distance as

                                                                                              x

                                                                                              x

                                                                                              yxyxyxyxd t

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Inner (Dot Scalar) Product

                                                                                              bull for Vectors and

                                                                                              yxyxyx td

                                                                                              jjj

                                                                                              1

                                                                                              x y

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Inner (Dot Scalar) Product

                                                                                              bull for Vectors and

                                                                                              bull Related to Norm via

                                                                                              yxyxyx td

                                                                                              jjj

                                                                                              1

                                                                                              x y

                                                                                              21 xxx

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                              bull measures ldquoangle between and rdquo as

                                                                                              yyxx

                                                                                              yx

                                                                                              yx

                                                                                              yxyxangle

                                                                                              tt

                                                                                              t

                                                                                              11 cos

                                                                                              cos

                                                                                              x y

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                              bull measures ldquoangle between and rdquo as

                                                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                                                              if and only if

                                                                                              yyxx

                                                                                              yx

                                                                                              yx

                                                                                              yxyxangle

                                                                                              tt

                                                                                              t

                                                                                              11 cos

                                                                                              cos

                                                                                              x y

                                                                                              yx 0 yx

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis

                                                                                              bull All Orthogonal to each other

                                                                                              ie for

                                                                                              bull All have Length 1

                                                                                              ie for

                                                                                              nvv 1

                                                                                              1 ii vv

                                                                                              0 ii vv ii

                                                                                              ni 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis (cont)

                                                                                              bull Spectral Representation

                                                                                              where

                                                                                              nvv 1

                                                                                              n

                                                                                              iii vax

                                                                                              1

                                                                                              ii vxa

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis (cont)

                                                                                              bull Spectral Representation

                                                                                              where

                                                                                              (Coefficient is Inner Product Cool Notation)

                                                                                              nvv 1

                                                                                              n

                                                                                              iii vax

                                                                                              1

                                                                                              ii vxa

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis (cont)

                                                                                              bull Spectral Representation

                                                                                              where

                                                                                              Check

                                                                                              nvv 1

                                                                                              n

                                                                                              iii vax

                                                                                              1

                                                                                              ii vxa

                                                                                              iii

                                                                                              n

                                                                                              iii

                                                                                              n

                                                                                              iiii avvavvavx

                                                                                              1

                                                                                              1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis (cont)

                                                                                              bull Spectral Representation

                                                                                              where

                                                                                              Check

                                                                                              bull Matrix Notation where ie

                                                                                              For the Basis Matrix

                                                                                              nvv 1

                                                                                              n

                                                                                              iii vax

                                                                                              1

                                                                                              ii vxa

                                                                                              iii

                                                                                              n

                                                                                              iii

                                                                                              n

                                                                                              iiii avvavvavx

                                                                                              1

                                                                                              1

                                                                                              aBx Bxa tt xBa t

                                                                                              nvvB 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Orthonormal Basis (cont)

                                                                                              bull Spectral Representation

                                                                                              where

                                                                                              Check

                                                                                              bull Matrix Notation where ie

                                                                                              is called transform of

                                                                                              (eg Fourier or Wavelet)

                                                                                              nvv 1

                                                                                              n

                                                                                              iii vax

                                                                                              1

                                                                                              ii vxa

                                                                                              iii

                                                                                              n

                                                                                              iii

                                                                                              n

                                                                                              iiii avvavvavx

                                                                                              1

                                                                                              1

                                                                                              aBx Bxa tt xBa t

                                                                                              xa

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Parseval identity for

                                                                                              in subsp genrsquod by o n basis

                                                                                              x

                                                                                              nvv 1

                                                                                              2

                                                                                              1

                                                                                              22

                                                                                              1

                                                                                              2 aavxx

                                                                                              n

                                                                                              ii

                                                                                              n

                                                                                              ii

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Parseval identity for

                                                                                              in subsp genrsquod by o n basis

                                                                                              bull Pythagorean theorem

                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                              bull ANOVA - sums of squares

                                                                                              x

                                                                                              nvv 1

                                                                                              2

                                                                                              1

                                                                                              22

                                                                                              1

                                                                                              2 aavxx

                                                                                              n

                                                                                              ii

                                                                                              n

                                                                                              ii

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Parseval identity for

                                                                                              in subsp genrsquod by o n basis

                                                                                              bull Pythagorean theorem

                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                              bull ANOVA - sums of squares

                                                                                              bull Transform has same length as

                                                                                              ie ldquorotation in rdquo

                                                                                              x

                                                                                              nvv 1

                                                                                              2

                                                                                              1

                                                                                              22

                                                                                              1

                                                                                              2 aavxx

                                                                                              n

                                                                                              ii

                                                                                              n

                                                                                              ii

                                                                                              a xd

                                                                                              Projection of a Vector onto a Subspace

                                                                                              bull Idea Member of that is Closest to

                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                              Review of Linear Algebra (Cont)x

                                                                                              xV

                                                                                              V

                                                                                              Projection of a Vector onto a Subspace

                                                                                              bull Idea Member of that is Closest to

                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                              bull Find that Solves

                                                                                              (ldquoLeast Squaresrdquo)

                                                                                              Review of Linear Algebra (Cont)x

                                                                                              xV

                                                                                              V

                                                                                              VxPV vxVv

                                                                                              min

                                                                                              Projection of a Vector onto a Subspace

                                                                                              bull Idea Member of that is Closest to

                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                              bull Find that Solves

                                                                                              (ldquoLeast Squaresrdquo)

                                                                                              bull For Inner Product (Hilbert) Space

                                                                                              Exists and is Unique

                                                                                              Review of Linear Algebra (Cont)x

                                                                                              xV

                                                                                              V

                                                                                              VxPV vxVv

                                                                                              min

                                                                                              xPV

                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                              bull General Solution in for Basis Matrix

                                                                                              Review of Linear Algebra (Cont)

                                                                                              d VB

                                                                                              xBBBBxP tVV

                                                                                              tVVV

                                                                                              1

                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                              bull General Solution in for Basis Matrix

                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                              (thus projection is another linear operation)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              d VB

                                                                                              xBBBBxP tVV

                                                                                              tVVV

                                                                                              1

                                                                                              tVV

                                                                                              tVVV BBBBP

                                                                                              1

                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                              bull General Solution in for Basis Matrix

                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                              (thus projection is another linear operation)

                                                                                              (note same operation underlies least squares)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              d VB

                                                                                              xBBBBxP tVV

                                                                                              tVVV

                                                                                              1

                                                                                              tVV

                                                                                              tVVV BBBBP

                                                                                              1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Projection using Orthonormal Basis

                                                                                              bull Basis Matrix is Orthonormal

                                                                                              nnVtV IBB

                                                                                              10

                                                                                              01

                                                                                              1

                                                                                              111

                                                                                              1

                                                                                              1

                                                                                              nnn

                                                                                              n

                                                                                              ntn

                                                                                              t

                                                                                              vvvv

                                                                                              vvvv

                                                                                              vv

                                                                                              v

                                                                                              v

                                                                                              nvv 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Projection using Orthonormal Basis

                                                                                              bull Basis Matrix is Orthonormal

                                                                                              bull So =

                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                              (Recall Right Multrsquon)

                                                                                              nnVtV IBB

                                                                                              10

                                                                                              01

                                                                                              1

                                                                                              111

                                                                                              1

                                                                                              1

                                                                                              nnn

                                                                                              n

                                                                                              ntn

                                                                                              t

                                                                                              vvvv

                                                                                              vvvv

                                                                                              vv

                                                                                              v

                                                                                              v

                                                                                              xBBxP tVVV

                                                                                              x V

                                                                                              nvv 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Projection using Orthonormal Basis (cont)

                                                                                              bull For Orthogonal Complement

                                                                                              and

                                                                                              V

                                                                                              xPxPx VV 222xPxPx VV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Projection using Orthonormal Basis (cont)

                                                                                              bull For Orthogonal Complement

                                                                                              and

                                                                                              bull Parseval Inequality

                                                                                              V

                                                                                              xPxPx VV 222xPxPx VV

                                                                                              2

                                                                                              1

                                                                                              22

                                                                                              1

                                                                                              22 aavxxxP

                                                                                              n

                                                                                              ii

                                                                                              n

                                                                                              iiV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              (Real) Unitary Matrices with

                                                                                              bull Orthonormal Basis Matrix

                                                                                              (So All of Above Applies)

                                                                                              ddU IUU t

                                                                                              Review of Linear Algebra (Cont)

                                                                                              (Real) Unitary Matrices with

                                                                                              bull Orthonormal Basis Matrix

                                                                                              (So All of Above Applies)

                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                              ddU IUU t

                                                                                              yxdyxyxyxUyUxUdn

                                                                                              i ii 2

                                                                                              1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              (Real) Unitary Matrices with

                                                                                              bull Orthonormal Basis Matrix

                                                                                              (So All of Above Applies)

                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                              ddU IUU t

                                                                                              yxdyxyxyxUyUxUdn

                                                                                              i ii 2

                                                                                              1

                                                                                              U

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Singular Value Decomposition (SVD)

                                                                                              For a Matrix

                                                                                              Find

                                                                                              ndX

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Singular Value Decomposition (SVD)

                                                                                              For a Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              with Entries

                                                                                              called Singular Values

                                                                                              ndX

                                                                                              ndS

                                                                                              )min(1 ndss

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Singular Value Decomposition (SVD)

                                                                                              For a Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              with Entries

                                                                                              called Singular Values

                                                                                              And Unitary (Rotation) Matrices

                                                                                              (recall )

                                                                                              ndX

                                                                                              ndS

                                                                                              )min(1 ndss

                                                                                              ddU nnV

                                                                                              IVVUU tt

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Singular Value Decomposition (SVD)

                                                                                              For a Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              with Entries

                                                                                              called Singular Values

                                                                                              And Unitary (Rotation) Matrices

                                                                                              (recall )

                                                                                              So That

                                                                                              ndX

                                                                                              ndS

                                                                                              )min(1 ndss

                                                                                              ddU nnV

                                                                                              IVVUU tt tUSVX

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Intuition behind Singular Value Decomposition

                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                              vVSUvVSUvX tt

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Intuition behind Singular Value Decomposition

                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                              bull First rotate

                                                                                              X

                                                                                              vVSUvVSUvX tt

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Intuition behind Singular Value Decomposition

                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                              bull First rotate

                                                                                              bull Second rescale coordinate axes (by )

                                                                                              X

                                                                                              vVSUvVSUvX tt

                                                                                              is

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Intuition behind Singular Value Decomposition

                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                              bull First rotate

                                                                                              bull Second rescale coordinate axes (by )

                                                                                              bull Third rotate again

                                                                                              X

                                                                                              vVSUvVSUvX tt

                                                                                              is

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Intuition behind Singular Value Decomposition

                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                              bull First rotate

                                                                                              bull Second rescale coordinate axes (by )

                                                                                              bull Third rotate again

                                                                                              bull ie have diagonalized the transformation

                                                                                              X

                                                                                              vVSUvVSUvX tt

                                                                                              is

                                                                                              SVD Compact Representation

                                                                                              Useful Labeling

                                                                                              Singular Values in Increasing Order

                                                                                              Review of Linear Algebra (Cont)

                                                                                              )min(1 dnss

                                                                                              SVD Compact Representation

                                                                                              Useful Labeling

                                                                                              Singular Values in Increasing Order

                                                                                              Note singular values = 0 can be omitted

                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              )min(1 dnss

                                                                                              Review of Linear Algebra (Cont)

                                                                                              r

                                                                                              SVD Compact Representation

                                                                                              Useful Labeling

                                                                                              Singular Values in Increasing Order

                                                                                              Note singular values = 0 can be omitted

                                                                                              Let = of positive singular values

                                                                                              )min(1 dnss

                                                                                              Review of Linear Algebra (Cont)

                                                                                              r

                                                                                              SVD Compact Representation

                                                                                              Useful Labeling

                                                                                              Singular Values in Increasing Order

                                                                                              Note singular values = 0 can be omitted

                                                                                              Let = of positive singular values

                                                                                              Then

                                                                                              Where are truncations of

                                                                                              trnrrrd VSUX

                                                                                              VSU

                                                                                              )min(1 dnss

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Full Representation

                                                                                              =

                                                                                              Graphics Display Assumes

                                                                                              ndX ddU ndS nn

                                                                                              tV

                                                                                              nd

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Full Representation

                                                                                              =

                                                                                              Full Rank Basis Matrix

                                                                                              ndX ddU ndS nn

                                                                                              tV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Full Representation

                                                                                              =

                                                                                              Full Rank Basis Matrix

                                                                                              All 0s in Bottom

                                                                                              ndX ddU ndS nn

                                                                                              tV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Reduced Representation

                                                                                              =

                                                                                              These Columns Get 0ed Out

                                                                                              ndX ddU nnS nn

                                                                                              tV

                                                                                              nnd 0

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Reduced Representation

                                                                                              =ndX ndU nnS nn

                                                                                              tV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Reduced Representation

                                                                                              =

                                                                                              Also Some of These May be 0

                                                                                              ndX ndU nnS nn

                                                                                              tV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Compact Representation

                                                                                              = ndX rdU

                                                                                              rrS nrtV

                                                                                              0

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Compact Representation

                                                                                              =

                                                                                              These Get 0ed Out

                                                                                              ndX rdU

                                                                                              rrS nrtV

                                                                                              0

                                                                                              Review of Linear Algebra (Cont)

                                                                                              SVD Compact Representation

                                                                                              = ndX rdU

                                                                                              rrS nrtV

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition

                                                                                              For a (Symmetric) Square Matrix

                                                                                              Find

                                                                                              ddX

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition

                                                                                              For a (Symmetric) Square Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              Called Eigenvalues

                                                                                              Convenient Ordering

                                                                                              ddX

                                                                                              d

                                                                                              D

                                                                                              0

                                                                                              01

                                                                                              n 1

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition

                                                                                              For a (Symmetric) Square Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              And an Orthonormal Matrix

                                                                                              (ie )

                                                                                              ddX

                                                                                              d

                                                                                              D

                                                                                              0

                                                                                              01

                                                                                              ddB

                                                                                              ddtt IBBBB

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition

                                                                                              For a (Symmetric) Square Matrix

                                                                                              Find a Diagonal Matrix

                                                                                              And an Orthonormal Matrix

                                                                                              (ie )

                                                                                              So that ie

                                                                                              ddX

                                                                                              d

                                                                                              D

                                                                                              0

                                                                                              01

                                                                                              ddB

                                                                                              ddtt IBBBB

                                                                                              DBBX tBDBX

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                              (looks similar)

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                              Complex (uses )

                                                                                              VU

                                                                                              1i

                                                                                              Review of Linear Algebra (Cont)

                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                              VU

                                                                                              X

                                                                                              BVU

                                                                                              1i

                                                                                              • Participant Presentations
                                                                                              • PCA to find clusters
                                                                                              • Statistical Smoothing
                                                                                              • Kernel Density Estimation
                                                                                              • Scatterplot Smoothing
                                                                                              • Statistical Smoothing (2)
                                                                                              • SiZer Background
                                                                                              • SiZer Background (2)
                                                                                              • SiZer Background (3)
                                                                                              • SiZer Background (4)
                                                                                              • SiZer Background (5)
                                                                                              • SiZer Overview
                                                                                              • PCA to find clusters (2)
                                                                                              • PCA to find clusters (3)
                                                                                              • PCA to find clusters (4)
                                                                                              • PCA to find clusters (5)
                                                                                              • PCA to find clusters (6)
                                                                                              • PCA to find clusters (7)
                                                                                              • Recall Yeast Cell Cycle Data
                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                              • Frequency 2 Analysis
                                                                                              • Frequency 2 Analysis (2)
                                                                                              • Frequency 2 Analysis (3)
                                                                                              • Yeast Cell Cycle
                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                              • Reclassification of Major Genes
                                                                                              • Compare to Previous Classifrsquon
                                                                                              • New Subpopulation View
                                                                                              • New Subpopulation View (2)
                                                                                              • Detailed Look at PCA
                                                                                              • PCA Rediscovery ndash Renaming
                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                              • An Interesting Historical Note
                                                                                              • An Interesting Historical Note (2)
                                                                                              • Detailed Look at PCA (2)
                                                                                              • Detailed Look at PCA (3)
                                                                                              • Review of Linear Algebra
                                                                                              • Review of Linear Algebra (2)
                                                                                              • Review of Linear Algebra (3)
                                                                                              • Review of Linear Algebra (Cont)
                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Origin

                                                                                                Note Planes not Through the Origin

                                                                                                are not Subspaces

                                                                                                (Do not Contain )00 x

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                                (all Linear Combos of them =

                                                                                                = Containing Hyperplane

                                                                                                through Origin)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Basis of Subspace Set of Vectors that

                                                                                                bull Span ie Everything is a Lin Com of them

                                                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Basis of Subspace Set of Vectors that

                                                                                                bull Span ie Everything is a Lin Com of them

                                                                                                bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                bull eg ldquoUnit Vector Basisrdquo

                                                                                                bull Since

                                                                                                d

                                                                                                1

                                                                                                0

                                                                                                0

                                                                                                0

                                                                                                1

                                                                                                0

                                                                                                0

                                                                                                0

                                                                                                1

                                                                                                1

                                                                                                0

                                                                                                0

                                                                                                0

                                                                                                1

                                                                                                0

                                                                                                0

                                                                                                0

                                                                                                1

                                                                                                212

                                                                                                1

                                                                                                d

                                                                                                d

                                                                                                xxx

                                                                                                x

                                                                                                x

                                                                                                x

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Basis Matrix of subspace of

                                                                                                Given a basis

                                                                                                create matrix of columns

                                                                                                dnvv 1

                                                                                                nddnd

                                                                                                n

                                                                                                n

                                                                                                vv

                                                                                                vv

                                                                                                vvB

                                                                                                1

                                                                                                111

                                                                                                1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                                where

                                                                                                n

                                                                                                iii aBva

                                                                                                1

                                                                                                na

                                                                                                a

                                                                                                a 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                                where

                                                                                                Note Right Multiplication Gives

                                                                                                Linear Combination of Column Vectors

                                                                                                n

                                                                                                iii aBva

                                                                                                1

                                                                                                na

                                                                                                a

                                                                                                a 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Then linear combo is a matrix multiplicatrsquon

                                                                                                where

                                                                                                Check sizes

                                                                                                n

                                                                                                iii aBva

                                                                                                1

                                                                                                na

                                                                                                a

                                                                                                a 1

                                                                                                )1()(1 nndd

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                For matrices

                                                                                                Define the Matrix Product

                                                                                                mkk

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                nmm

                                                                                                n

                                                                                                bb

                                                                                                bb

                                                                                                B

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iniik

                                                                                                m

                                                                                                iiik

                                                                                                m

                                                                                                inii

                                                                                                m

                                                                                                iii

                                                                                                baba

                                                                                                baba

                                                                                                AB

                                                                                                1

                                                                                                11

                                                                                                11

                                                                                                111

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                For matrices

                                                                                                Define the Matrix Product

                                                                                                (Inner Products of Rows With Columns )

                                                                                                mkk

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                nmm

                                                                                                n

                                                                                                bb

                                                                                                bb

                                                                                                B

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iniik

                                                                                                m

                                                                                                iiik

                                                                                                m

                                                                                                inii

                                                                                                m

                                                                                                iii

                                                                                                baba

                                                                                                baba

                                                                                                AB

                                                                                                1

                                                                                                11

                                                                                                11

                                                                                                111

                                                                                                A B

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                For matrices

                                                                                                Define the Matrix Product

                                                                                                (Inner Products of Rows With Columns )

                                                                                                (Composition of Linear Transformations)

                                                                                                mkk

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                nmm

                                                                                                n

                                                                                                bb

                                                                                                bb

                                                                                                B

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iniik

                                                                                                m

                                                                                                iiik

                                                                                                m

                                                                                                inii

                                                                                                m

                                                                                                iii

                                                                                                baba

                                                                                                baba

                                                                                                AB

                                                                                                1

                                                                                                11

                                                                                                11

                                                                                                111

                                                                                                A B

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                For matrices

                                                                                                Define the Matrix Product

                                                                                                (Inner Products of Rows With Columns )

                                                                                                (Composition of Linear Transformations)

                                                                                                Often Useful to Check Sizes

                                                                                                mkk

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                nmm

                                                                                                n

                                                                                                bb

                                                                                                bb

                                                                                                B

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iniik

                                                                                                m

                                                                                                iiik

                                                                                                m

                                                                                                inii

                                                                                                m

                                                                                                iii

                                                                                                baba

                                                                                                baba

                                                                                                AB

                                                                                                1

                                                                                                11

                                                                                                11

                                                                                                111

                                                                                                nmmknk

                                                                                                A B

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                For matrices

                                                                                                Define the Matrix Product

                                                                                                (Inner Products of Rows With Columns )

                                                                                                (Composition of Linear Transformations)

                                                                                                Often Useful to Check Sizes

                                                                                                mkk

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                nmm

                                                                                                n

                                                                                                bb

                                                                                                bb

                                                                                                B

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iniik

                                                                                                m

                                                                                                iiik

                                                                                                m

                                                                                                inii

                                                                                                m

                                                                                                iii

                                                                                                baba

                                                                                                baba

                                                                                                AB

                                                                                                1

                                                                                                11

                                                                                                11

                                                                                                111

                                                                                                nmmknk

                                                                                                A B

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Matrix Trace

                                                                                                bull For a Square Matrix

                                                                                                bull Define

                                                                                                mmm

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iiiaAtr

                                                                                                1)(

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Matrix Trace

                                                                                                bull For a Square Matrix

                                                                                                bull Define

                                                                                                bull Trace Commutes with Matrix Multiplication

                                                                                                mmm

                                                                                                m

                                                                                                aa

                                                                                                aa

                                                                                                A

                                                                                                1

                                                                                                111

                                                                                                m

                                                                                                iiiaAtr

                                                                                                1)(

                                                                                                BAtrABtr

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                bull (Use Basis Above)

                                                                                                bull eg dim of a line is 1

                                                                                                bull eg dim of a plane is 2

                                                                                                dd dim

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                bull (Use Basis Above)

                                                                                                bull eg dim of a line is 1

                                                                                                bull eg dim of a plane is 2

                                                                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                (in Statistical Uses eg ANOVA)

                                                                                                dd dim

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Norm of a Vector

                                                                                                bull in d 21

                                                                                                21

                                                                                                1

                                                                                                2 xxxx td

                                                                                                jj

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Norm of a Vector

                                                                                                bull in

                                                                                                bull Idea length of the vector

                                                                                                d 2121

                                                                                                1

                                                                                                2 xxxx td

                                                                                                jj

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Norm of a Vector

                                                                                                bull in

                                                                                                bull Idea length of the vector

                                                                                                bull Note strange properties for high

                                                                                                eg ldquolength of diagonal of unit cuberdquo =

                                                                                                d 2121

                                                                                                1

                                                                                                2 xxxx td

                                                                                                jj

                                                                                                d

                                                                                                d

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Norm of a Vector (cont)

                                                                                                bull Length Normalized Vector

                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                amp is a Direction Vector)

                                                                                                x

                                                                                                x

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Norm of a Vector (cont)

                                                                                                bull Length Normalized Vector

                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                amp is a Direction Vector)

                                                                                                bull Define Distance as

                                                                                                x

                                                                                                x

                                                                                                yxyxyxyxd t

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Inner (Dot Scalar) Product

                                                                                                bull for Vectors and

                                                                                                yxyxyx td

                                                                                                jjj

                                                                                                1

                                                                                                x y

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Inner (Dot Scalar) Product

                                                                                                bull for Vectors and

                                                                                                bull Related to Norm via

                                                                                                yxyxyx td

                                                                                                jjj

                                                                                                1

                                                                                                x y

                                                                                                21 xxx

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                yyxx

                                                                                                yx

                                                                                                yx

                                                                                                yxyxangle

                                                                                                tt

                                                                                                t

                                                                                                11 cos

                                                                                                cos

                                                                                                x y

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                if and only if

                                                                                                yyxx

                                                                                                yx

                                                                                                yx

                                                                                                yxyxangle

                                                                                                tt

                                                                                                t

                                                                                                11 cos

                                                                                                cos

                                                                                                x y

                                                                                                yx 0 yx

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis

                                                                                                bull All Orthogonal to each other

                                                                                                ie for

                                                                                                bull All have Length 1

                                                                                                ie for

                                                                                                nvv 1

                                                                                                1 ii vv

                                                                                                0 ii vv ii

                                                                                                ni 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis (cont)

                                                                                                bull Spectral Representation

                                                                                                where

                                                                                                nvv 1

                                                                                                n

                                                                                                iii vax

                                                                                                1

                                                                                                ii vxa

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis (cont)

                                                                                                bull Spectral Representation

                                                                                                where

                                                                                                (Coefficient is Inner Product Cool Notation)

                                                                                                nvv 1

                                                                                                n

                                                                                                iii vax

                                                                                                1

                                                                                                ii vxa

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis (cont)

                                                                                                bull Spectral Representation

                                                                                                where

                                                                                                Check

                                                                                                nvv 1

                                                                                                n

                                                                                                iii vax

                                                                                                1

                                                                                                ii vxa

                                                                                                iii

                                                                                                n

                                                                                                iii

                                                                                                n

                                                                                                iiii avvavvavx

                                                                                                1

                                                                                                1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis (cont)

                                                                                                bull Spectral Representation

                                                                                                where

                                                                                                Check

                                                                                                bull Matrix Notation where ie

                                                                                                For the Basis Matrix

                                                                                                nvv 1

                                                                                                n

                                                                                                iii vax

                                                                                                1

                                                                                                ii vxa

                                                                                                iii

                                                                                                n

                                                                                                iii

                                                                                                n

                                                                                                iiii avvavvavx

                                                                                                1

                                                                                                1

                                                                                                aBx Bxa tt xBa t

                                                                                                nvvB 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Orthonormal Basis (cont)

                                                                                                bull Spectral Representation

                                                                                                where

                                                                                                Check

                                                                                                bull Matrix Notation where ie

                                                                                                is called transform of

                                                                                                (eg Fourier or Wavelet)

                                                                                                nvv 1

                                                                                                n

                                                                                                iii vax

                                                                                                1

                                                                                                ii vxa

                                                                                                iii

                                                                                                n

                                                                                                iii

                                                                                                n

                                                                                                iiii avvavvavx

                                                                                                1

                                                                                                1

                                                                                                aBx Bxa tt xBa t

                                                                                                xa

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Parseval identity for

                                                                                                in subsp genrsquod by o n basis

                                                                                                x

                                                                                                nvv 1

                                                                                                2

                                                                                                1

                                                                                                22

                                                                                                1

                                                                                                2 aavxx

                                                                                                n

                                                                                                ii

                                                                                                n

                                                                                                ii

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Parseval identity for

                                                                                                in subsp genrsquod by o n basis

                                                                                                bull Pythagorean theorem

                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                bull ANOVA - sums of squares

                                                                                                x

                                                                                                nvv 1

                                                                                                2

                                                                                                1

                                                                                                22

                                                                                                1

                                                                                                2 aavxx

                                                                                                n

                                                                                                ii

                                                                                                n

                                                                                                ii

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Parseval identity for

                                                                                                in subsp genrsquod by o n basis

                                                                                                bull Pythagorean theorem

                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                bull ANOVA - sums of squares

                                                                                                bull Transform has same length as

                                                                                                ie ldquorotation in rdquo

                                                                                                x

                                                                                                nvv 1

                                                                                                2

                                                                                                1

                                                                                                22

                                                                                                1

                                                                                                2 aavxx

                                                                                                n

                                                                                                ii

                                                                                                n

                                                                                                ii

                                                                                                a xd

                                                                                                Projection of a Vector onto a Subspace

                                                                                                bull Idea Member of that is Closest to

                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                Review of Linear Algebra (Cont)x

                                                                                                xV

                                                                                                V

                                                                                                Projection of a Vector onto a Subspace

                                                                                                bull Idea Member of that is Closest to

                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                bull Find that Solves

                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                Review of Linear Algebra (Cont)x

                                                                                                xV

                                                                                                V

                                                                                                VxPV vxVv

                                                                                                min

                                                                                                Projection of a Vector onto a Subspace

                                                                                                bull Idea Member of that is Closest to

                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                bull Find that Solves

                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                bull For Inner Product (Hilbert) Space

                                                                                                Exists and is Unique

                                                                                                Review of Linear Algebra (Cont)x

                                                                                                xV

                                                                                                V

                                                                                                VxPV vxVv

                                                                                                min

                                                                                                xPV

                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                bull General Solution in for Basis Matrix

                                                                                                Review of Linear Algebra (Cont)

                                                                                                d VB

                                                                                                xBBBBxP tVV

                                                                                                tVVV

                                                                                                1

                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                bull General Solution in for Basis Matrix

                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                (thus projection is another linear operation)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                d VB

                                                                                                xBBBBxP tVV

                                                                                                tVVV

                                                                                                1

                                                                                                tVV

                                                                                                tVVV BBBBP

                                                                                                1

                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                bull General Solution in for Basis Matrix

                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                (thus projection is another linear operation)

                                                                                                (note same operation underlies least squares)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                d VB

                                                                                                xBBBBxP tVV

                                                                                                tVVV

                                                                                                1

                                                                                                tVV

                                                                                                tVVV BBBBP

                                                                                                1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Projection using Orthonormal Basis

                                                                                                bull Basis Matrix is Orthonormal

                                                                                                nnVtV IBB

                                                                                                10

                                                                                                01

                                                                                                1

                                                                                                111

                                                                                                1

                                                                                                1

                                                                                                nnn

                                                                                                n

                                                                                                ntn

                                                                                                t

                                                                                                vvvv

                                                                                                vvvv

                                                                                                vv

                                                                                                v

                                                                                                v

                                                                                                nvv 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Projection using Orthonormal Basis

                                                                                                bull Basis Matrix is Orthonormal

                                                                                                bull So =

                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                (Recall Right Multrsquon)

                                                                                                nnVtV IBB

                                                                                                10

                                                                                                01

                                                                                                1

                                                                                                111

                                                                                                1

                                                                                                1

                                                                                                nnn

                                                                                                n

                                                                                                ntn

                                                                                                t

                                                                                                vvvv

                                                                                                vvvv

                                                                                                vv

                                                                                                v

                                                                                                v

                                                                                                xBBxP tVVV

                                                                                                x V

                                                                                                nvv 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                bull For Orthogonal Complement

                                                                                                and

                                                                                                V

                                                                                                xPxPx VV 222xPxPx VV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                bull For Orthogonal Complement

                                                                                                and

                                                                                                bull Parseval Inequality

                                                                                                V

                                                                                                xPxPx VV 222xPxPx VV

                                                                                                2

                                                                                                1

                                                                                                22

                                                                                                1

                                                                                                22 aavxxxP

                                                                                                n

                                                                                                ii

                                                                                                n

                                                                                                iiV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                (Real) Unitary Matrices with

                                                                                                bull Orthonormal Basis Matrix

                                                                                                (So All of Above Applies)

                                                                                                ddU IUU t

                                                                                                Review of Linear Algebra (Cont)

                                                                                                (Real) Unitary Matrices with

                                                                                                bull Orthonormal Basis Matrix

                                                                                                (So All of Above Applies)

                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                ddU IUU t

                                                                                                yxdyxyxyxUyUxUdn

                                                                                                i ii 2

                                                                                                1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                (Real) Unitary Matrices with

                                                                                                bull Orthonormal Basis Matrix

                                                                                                (So All of Above Applies)

                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                ddU IUU t

                                                                                                yxdyxyxyxUyUxUdn

                                                                                                i ii 2

                                                                                                1

                                                                                                U

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Singular Value Decomposition (SVD)

                                                                                                For a Matrix

                                                                                                Find

                                                                                                ndX

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Singular Value Decomposition (SVD)

                                                                                                For a Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                with Entries

                                                                                                called Singular Values

                                                                                                ndX

                                                                                                ndS

                                                                                                )min(1 ndss

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Singular Value Decomposition (SVD)

                                                                                                For a Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                with Entries

                                                                                                called Singular Values

                                                                                                And Unitary (Rotation) Matrices

                                                                                                (recall )

                                                                                                ndX

                                                                                                ndS

                                                                                                )min(1 ndss

                                                                                                ddU nnV

                                                                                                IVVUU tt

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Singular Value Decomposition (SVD)

                                                                                                For a Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                with Entries

                                                                                                called Singular Values

                                                                                                And Unitary (Rotation) Matrices

                                                                                                (recall )

                                                                                                So That

                                                                                                ndX

                                                                                                ndS

                                                                                                )min(1 ndss

                                                                                                ddU nnV

                                                                                                IVVUU tt tUSVX

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Intuition behind Singular Value Decomposition

                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                vVSUvVSUvX tt

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Intuition behind Singular Value Decomposition

                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                bull First rotate

                                                                                                X

                                                                                                vVSUvVSUvX tt

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Intuition behind Singular Value Decomposition

                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                bull First rotate

                                                                                                bull Second rescale coordinate axes (by )

                                                                                                X

                                                                                                vVSUvVSUvX tt

                                                                                                is

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Intuition behind Singular Value Decomposition

                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                bull First rotate

                                                                                                bull Second rescale coordinate axes (by )

                                                                                                bull Third rotate again

                                                                                                X

                                                                                                vVSUvVSUvX tt

                                                                                                is

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Intuition behind Singular Value Decomposition

                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                bull First rotate

                                                                                                bull Second rescale coordinate axes (by )

                                                                                                bull Third rotate again

                                                                                                bull ie have diagonalized the transformation

                                                                                                X

                                                                                                vVSUvVSUvX tt

                                                                                                is

                                                                                                SVD Compact Representation

                                                                                                Useful Labeling

                                                                                                Singular Values in Increasing Order

                                                                                                Review of Linear Algebra (Cont)

                                                                                                )min(1 dnss

                                                                                                SVD Compact Representation

                                                                                                Useful Labeling

                                                                                                Singular Values in Increasing Order

                                                                                                Note singular values = 0 can be omitted

                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                )min(1 dnss

                                                                                                Review of Linear Algebra (Cont)

                                                                                                r

                                                                                                SVD Compact Representation

                                                                                                Useful Labeling

                                                                                                Singular Values in Increasing Order

                                                                                                Note singular values = 0 can be omitted

                                                                                                Let = of positive singular values

                                                                                                )min(1 dnss

                                                                                                Review of Linear Algebra (Cont)

                                                                                                r

                                                                                                SVD Compact Representation

                                                                                                Useful Labeling

                                                                                                Singular Values in Increasing Order

                                                                                                Note singular values = 0 can be omitted

                                                                                                Let = of positive singular values

                                                                                                Then

                                                                                                Where are truncations of

                                                                                                trnrrrd VSUX

                                                                                                VSU

                                                                                                )min(1 dnss

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Full Representation

                                                                                                =

                                                                                                Graphics Display Assumes

                                                                                                ndX ddU ndS nn

                                                                                                tV

                                                                                                nd

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Full Representation

                                                                                                =

                                                                                                Full Rank Basis Matrix

                                                                                                ndX ddU ndS nn

                                                                                                tV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Full Representation

                                                                                                =

                                                                                                Full Rank Basis Matrix

                                                                                                All 0s in Bottom

                                                                                                ndX ddU ndS nn

                                                                                                tV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Reduced Representation

                                                                                                =

                                                                                                These Columns Get 0ed Out

                                                                                                ndX ddU nnS nn

                                                                                                tV

                                                                                                nnd 0

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Reduced Representation

                                                                                                =ndX ndU nnS nn

                                                                                                tV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Reduced Representation

                                                                                                =

                                                                                                Also Some of These May be 0

                                                                                                ndX ndU nnS nn

                                                                                                tV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Compact Representation

                                                                                                = ndX rdU

                                                                                                rrS nrtV

                                                                                                0

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Compact Representation

                                                                                                =

                                                                                                These Get 0ed Out

                                                                                                ndX rdU

                                                                                                rrS nrtV

                                                                                                0

                                                                                                Review of Linear Algebra (Cont)

                                                                                                SVD Compact Representation

                                                                                                = ndX rdU

                                                                                                rrS nrtV

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition

                                                                                                For a (Symmetric) Square Matrix

                                                                                                Find

                                                                                                ddX

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition

                                                                                                For a (Symmetric) Square Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                Called Eigenvalues

                                                                                                Convenient Ordering

                                                                                                ddX

                                                                                                d

                                                                                                D

                                                                                                0

                                                                                                01

                                                                                                n 1

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition

                                                                                                For a (Symmetric) Square Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                And an Orthonormal Matrix

                                                                                                (ie )

                                                                                                ddX

                                                                                                d

                                                                                                D

                                                                                                0

                                                                                                01

                                                                                                ddB

                                                                                                ddtt IBBBB

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition

                                                                                                For a (Symmetric) Square Matrix

                                                                                                Find a Diagonal Matrix

                                                                                                And an Orthonormal Matrix

                                                                                                (ie )

                                                                                                So that ie

                                                                                                ddX

                                                                                                d

                                                                                                D

                                                                                                0

                                                                                                01

                                                                                                ddB

                                                                                                ddtt IBBBB

                                                                                                DBBX tBDBX

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                (looks similar)

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                Complex (uses )

                                                                                                VU

                                                                                                1i

                                                                                                Review of Linear Algebra (Cont)

                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                VU

                                                                                                X

                                                                                                BVU

                                                                                                1i

                                                                                                • Participant Presentations
                                                                                                • PCA to find clusters
                                                                                                • Statistical Smoothing
                                                                                                • Kernel Density Estimation
                                                                                                • Scatterplot Smoothing
                                                                                                • Statistical Smoothing (2)
                                                                                                • SiZer Background
                                                                                                • SiZer Background (2)
                                                                                                • SiZer Background (3)
                                                                                                • SiZer Background (4)
                                                                                                • SiZer Background (5)
                                                                                                • SiZer Overview
                                                                                                • PCA to find clusters (2)
                                                                                                • PCA to find clusters (3)
                                                                                                • PCA to find clusters (4)
                                                                                                • PCA to find clusters (5)
                                                                                                • PCA to find clusters (6)
                                                                                                • PCA to find clusters (7)
                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                • Frequency 2 Analysis
                                                                                                • Frequency 2 Analysis (2)
                                                                                                • Frequency 2 Analysis (3)
                                                                                                • Yeast Cell Cycle
                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                • Reclassification of Major Genes
                                                                                                • Compare to Previous Classifrsquon
                                                                                                • New Subpopulation View
                                                                                                • New Subpopulation View (2)
                                                                                                • Detailed Look at PCA
                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                • An Interesting Historical Note
                                                                                                • An Interesting Historical Note (2)
                                                                                                • Detailed Look at PCA (2)
                                                                                                • Detailed Look at PCA (3)
                                                                                                • Review of Linear Algebra
                                                                                                • Review of Linear Algebra (2)
                                                                                                • Review of Linear Algebra (3)
                                                                                                • Review of Linear Algebra (Cont)
                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Subspacebull Subset that is Again a Vector Spacebull ie Closed under Linear Combinationbull eg Lines through the Originbull eg Planes through the Originbull eg Subsp ldquoGenerated Byrdquo a Set of Vectors

                                                                                                  (all Linear Combos of them =

                                                                                                  = Containing Hyperplane

                                                                                                  through Origin)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Basis of Subspace Set of Vectors that

                                                                                                  bull Span ie Everything is a Lin Com of them

                                                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Basis of Subspace Set of Vectors that

                                                                                                  bull Span ie Everything is a Lin Com of them

                                                                                                  bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                  bull eg ldquoUnit Vector Basisrdquo

                                                                                                  bull Since

                                                                                                  d

                                                                                                  1

                                                                                                  0

                                                                                                  0

                                                                                                  0

                                                                                                  1

                                                                                                  0

                                                                                                  0

                                                                                                  0

                                                                                                  1

                                                                                                  1

                                                                                                  0

                                                                                                  0

                                                                                                  0

                                                                                                  1

                                                                                                  0

                                                                                                  0

                                                                                                  0

                                                                                                  1

                                                                                                  212

                                                                                                  1

                                                                                                  d

                                                                                                  d

                                                                                                  xxx

                                                                                                  x

                                                                                                  x

                                                                                                  x

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Basis Matrix of subspace of

                                                                                                  Given a basis

                                                                                                  create matrix of columns

                                                                                                  dnvv 1

                                                                                                  nddnd

                                                                                                  n

                                                                                                  n

                                                                                                  vv

                                                                                                  vv

                                                                                                  vvB

                                                                                                  1

                                                                                                  111

                                                                                                  1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                                  where

                                                                                                  n

                                                                                                  iii aBva

                                                                                                  1

                                                                                                  na

                                                                                                  a

                                                                                                  a 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                                  where

                                                                                                  Note Right Multiplication Gives

                                                                                                  Linear Combination of Column Vectors

                                                                                                  n

                                                                                                  iii aBva

                                                                                                  1

                                                                                                  na

                                                                                                  a

                                                                                                  a 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Then linear combo is a matrix multiplicatrsquon

                                                                                                  where

                                                                                                  Check sizes

                                                                                                  n

                                                                                                  iii aBva

                                                                                                  1

                                                                                                  na

                                                                                                  a

                                                                                                  a 1

                                                                                                  )1()(1 nndd

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                  For matrices

                                                                                                  Define the Matrix Product

                                                                                                  mkk

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  nmm

                                                                                                  n

                                                                                                  bb

                                                                                                  bb

                                                                                                  B

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iniik

                                                                                                  m

                                                                                                  iiik

                                                                                                  m

                                                                                                  inii

                                                                                                  m

                                                                                                  iii

                                                                                                  baba

                                                                                                  baba

                                                                                                  AB

                                                                                                  1

                                                                                                  11

                                                                                                  11

                                                                                                  111

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                  For matrices

                                                                                                  Define the Matrix Product

                                                                                                  (Inner Products of Rows With Columns )

                                                                                                  mkk

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  nmm

                                                                                                  n

                                                                                                  bb

                                                                                                  bb

                                                                                                  B

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iniik

                                                                                                  m

                                                                                                  iiik

                                                                                                  m

                                                                                                  inii

                                                                                                  m

                                                                                                  iii

                                                                                                  baba

                                                                                                  baba

                                                                                                  AB

                                                                                                  1

                                                                                                  11

                                                                                                  11

                                                                                                  111

                                                                                                  A B

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                  For matrices

                                                                                                  Define the Matrix Product

                                                                                                  (Inner Products of Rows With Columns )

                                                                                                  (Composition of Linear Transformations)

                                                                                                  mkk

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  nmm

                                                                                                  n

                                                                                                  bb

                                                                                                  bb

                                                                                                  B

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iniik

                                                                                                  m

                                                                                                  iiik

                                                                                                  m

                                                                                                  inii

                                                                                                  m

                                                                                                  iii

                                                                                                  baba

                                                                                                  baba

                                                                                                  AB

                                                                                                  1

                                                                                                  11

                                                                                                  11

                                                                                                  111

                                                                                                  A B

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                  For matrices

                                                                                                  Define the Matrix Product

                                                                                                  (Inner Products of Rows With Columns )

                                                                                                  (Composition of Linear Transformations)

                                                                                                  Often Useful to Check Sizes

                                                                                                  mkk

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  nmm

                                                                                                  n

                                                                                                  bb

                                                                                                  bb

                                                                                                  B

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iniik

                                                                                                  m

                                                                                                  iiik

                                                                                                  m

                                                                                                  inii

                                                                                                  m

                                                                                                  iii

                                                                                                  baba

                                                                                                  baba

                                                                                                  AB

                                                                                                  1

                                                                                                  11

                                                                                                  11

                                                                                                  111

                                                                                                  nmmknk

                                                                                                  A B

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                  For matrices

                                                                                                  Define the Matrix Product

                                                                                                  (Inner Products of Rows With Columns )

                                                                                                  (Composition of Linear Transformations)

                                                                                                  Often Useful to Check Sizes

                                                                                                  mkk

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  nmm

                                                                                                  n

                                                                                                  bb

                                                                                                  bb

                                                                                                  B

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iniik

                                                                                                  m

                                                                                                  iiik

                                                                                                  m

                                                                                                  inii

                                                                                                  m

                                                                                                  iii

                                                                                                  baba

                                                                                                  baba

                                                                                                  AB

                                                                                                  1

                                                                                                  11

                                                                                                  11

                                                                                                  111

                                                                                                  nmmknk

                                                                                                  A B

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Matrix Trace

                                                                                                  bull For a Square Matrix

                                                                                                  bull Define

                                                                                                  mmm

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iiiaAtr

                                                                                                  1)(

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Matrix Trace

                                                                                                  bull For a Square Matrix

                                                                                                  bull Define

                                                                                                  bull Trace Commutes with Matrix Multiplication

                                                                                                  mmm

                                                                                                  m

                                                                                                  aa

                                                                                                  aa

                                                                                                  A

                                                                                                  1

                                                                                                  111

                                                                                                  m

                                                                                                  iiiaAtr

                                                                                                  1)(

                                                                                                  BAtrABtr

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                  bull (Use Basis Above)

                                                                                                  bull eg dim of a line is 1

                                                                                                  bull eg dim of a plane is 2

                                                                                                  dd dim

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                  bull (Use Basis Above)

                                                                                                  bull eg dim of a line is 1

                                                                                                  bull eg dim of a plane is 2

                                                                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                  (in Statistical Uses eg ANOVA)

                                                                                                  dd dim

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Norm of a Vector

                                                                                                  bull in d 21

                                                                                                  21

                                                                                                  1

                                                                                                  2 xxxx td

                                                                                                  jj

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Norm of a Vector

                                                                                                  bull in

                                                                                                  bull Idea length of the vector

                                                                                                  d 2121

                                                                                                  1

                                                                                                  2 xxxx td

                                                                                                  jj

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Norm of a Vector

                                                                                                  bull in

                                                                                                  bull Idea length of the vector

                                                                                                  bull Note strange properties for high

                                                                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                                                                  d 2121

                                                                                                  1

                                                                                                  2 xxxx td

                                                                                                  jj

                                                                                                  d

                                                                                                  d

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Norm of a Vector (cont)

                                                                                                  bull Length Normalized Vector

                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                  amp is a Direction Vector)

                                                                                                  x

                                                                                                  x

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Norm of a Vector (cont)

                                                                                                  bull Length Normalized Vector

                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                  amp is a Direction Vector)

                                                                                                  bull Define Distance as

                                                                                                  x

                                                                                                  x

                                                                                                  yxyxyxyxd t

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Inner (Dot Scalar) Product

                                                                                                  bull for Vectors and

                                                                                                  yxyxyx td

                                                                                                  jjj

                                                                                                  1

                                                                                                  x y

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Inner (Dot Scalar) Product

                                                                                                  bull for Vectors and

                                                                                                  bull Related to Norm via

                                                                                                  yxyxyx td

                                                                                                  jjj

                                                                                                  1

                                                                                                  x y

                                                                                                  21 xxx

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                  yyxx

                                                                                                  yx

                                                                                                  yx

                                                                                                  yxyxangle

                                                                                                  tt

                                                                                                  t

                                                                                                  11 cos

                                                                                                  cos

                                                                                                  x y

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                  if and only if

                                                                                                  yyxx

                                                                                                  yx

                                                                                                  yx

                                                                                                  yxyxangle

                                                                                                  tt

                                                                                                  t

                                                                                                  11 cos

                                                                                                  cos

                                                                                                  x y

                                                                                                  yx 0 yx

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis

                                                                                                  bull All Orthogonal to each other

                                                                                                  ie for

                                                                                                  bull All have Length 1

                                                                                                  ie for

                                                                                                  nvv 1

                                                                                                  1 ii vv

                                                                                                  0 ii vv ii

                                                                                                  ni 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis (cont)

                                                                                                  bull Spectral Representation

                                                                                                  where

                                                                                                  nvv 1

                                                                                                  n

                                                                                                  iii vax

                                                                                                  1

                                                                                                  ii vxa

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis (cont)

                                                                                                  bull Spectral Representation

                                                                                                  where

                                                                                                  (Coefficient is Inner Product Cool Notation)

                                                                                                  nvv 1

                                                                                                  n

                                                                                                  iii vax

                                                                                                  1

                                                                                                  ii vxa

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis (cont)

                                                                                                  bull Spectral Representation

                                                                                                  where

                                                                                                  Check

                                                                                                  nvv 1

                                                                                                  n

                                                                                                  iii vax

                                                                                                  1

                                                                                                  ii vxa

                                                                                                  iii

                                                                                                  n

                                                                                                  iii

                                                                                                  n

                                                                                                  iiii avvavvavx

                                                                                                  1

                                                                                                  1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis (cont)

                                                                                                  bull Spectral Representation

                                                                                                  where

                                                                                                  Check

                                                                                                  bull Matrix Notation where ie

                                                                                                  For the Basis Matrix

                                                                                                  nvv 1

                                                                                                  n

                                                                                                  iii vax

                                                                                                  1

                                                                                                  ii vxa

                                                                                                  iii

                                                                                                  n

                                                                                                  iii

                                                                                                  n

                                                                                                  iiii avvavvavx

                                                                                                  1

                                                                                                  1

                                                                                                  aBx Bxa tt xBa t

                                                                                                  nvvB 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Orthonormal Basis (cont)

                                                                                                  bull Spectral Representation

                                                                                                  where

                                                                                                  Check

                                                                                                  bull Matrix Notation where ie

                                                                                                  is called transform of

                                                                                                  (eg Fourier or Wavelet)

                                                                                                  nvv 1

                                                                                                  n

                                                                                                  iii vax

                                                                                                  1

                                                                                                  ii vxa

                                                                                                  iii

                                                                                                  n

                                                                                                  iii

                                                                                                  n

                                                                                                  iiii avvavvavx

                                                                                                  1

                                                                                                  1

                                                                                                  aBx Bxa tt xBa t

                                                                                                  xa

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Parseval identity for

                                                                                                  in subsp genrsquod by o n basis

                                                                                                  x

                                                                                                  nvv 1

                                                                                                  2

                                                                                                  1

                                                                                                  22

                                                                                                  1

                                                                                                  2 aavxx

                                                                                                  n

                                                                                                  ii

                                                                                                  n

                                                                                                  ii

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Parseval identity for

                                                                                                  in subsp genrsquod by o n basis

                                                                                                  bull Pythagorean theorem

                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                  bull ANOVA - sums of squares

                                                                                                  x

                                                                                                  nvv 1

                                                                                                  2

                                                                                                  1

                                                                                                  22

                                                                                                  1

                                                                                                  2 aavxx

                                                                                                  n

                                                                                                  ii

                                                                                                  n

                                                                                                  ii

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Parseval identity for

                                                                                                  in subsp genrsquod by o n basis

                                                                                                  bull Pythagorean theorem

                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                  bull ANOVA - sums of squares

                                                                                                  bull Transform has same length as

                                                                                                  ie ldquorotation in rdquo

                                                                                                  x

                                                                                                  nvv 1

                                                                                                  2

                                                                                                  1

                                                                                                  22

                                                                                                  1

                                                                                                  2 aavxx

                                                                                                  n

                                                                                                  ii

                                                                                                  n

                                                                                                  ii

                                                                                                  a xd

                                                                                                  Projection of a Vector onto a Subspace

                                                                                                  bull Idea Member of that is Closest to

                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                  xV

                                                                                                  V

                                                                                                  Projection of a Vector onto a Subspace

                                                                                                  bull Idea Member of that is Closest to

                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                  bull Find that Solves

                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                  xV

                                                                                                  V

                                                                                                  VxPV vxVv

                                                                                                  min

                                                                                                  Projection of a Vector onto a Subspace

                                                                                                  bull Idea Member of that is Closest to

                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                  bull Find that Solves

                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                  bull For Inner Product (Hilbert) Space

                                                                                                  Exists and is Unique

                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                  xV

                                                                                                  V

                                                                                                  VxPV vxVv

                                                                                                  min

                                                                                                  xPV

                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                  bull General Solution in for Basis Matrix

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  d VB

                                                                                                  xBBBBxP tVV

                                                                                                  tVVV

                                                                                                  1

                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                  bull General Solution in for Basis Matrix

                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                  (thus projection is another linear operation)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  d VB

                                                                                                  xBBBBxP tVV

                                                                                                  tVVV

                                                                                                  1

                                                                                                  tVV

                                                                                                  tVVV BBBBP

                                                                                                  1

                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                  bull General Solution in for Basis Matrix

                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                  (thus projection is another linear operation)

                                                                                                  (note same operation underlies least squares)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  d VB

                                                                                                  xBBBBxP tVV

                                                                                                  tVVV

                                                                                                  1

                                                                                                  tVV

                                                                                                  tVVV BBBBP

                                                                                                  1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Projection using Orthonormal Basis

                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                  nnVtV IBB

                                                                                                  10

                                                                                                  01

                                                                                                  1

                                                                                                  111

                                                                                                  1

                                                                                                  1

                                                                                                  nnn

                                                                                                  n

                                                                                                  ntn

                                                                                                  t

                                                                                                  vvvv

                                                                                                  vvvv

                                                                                                  vv

                                                                                                  v

                                                                                                  v

                                                                                                  nvv 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Projection using Orthonormal Basis

                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                  bull So =

                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                  (Recall Right Multrsquon)

                                                                                                  nnVtV IBB

                                                                                                  10

                                                                                                  01

                                                                                                  1

                                                                                                  111

                                                                                                  1

                                                                                                  1

                                                                                                  nnn

                                                                                                  n

                                                                                                  ntn

                                                                                                  t

                                                                                                  vvvv

                                                                                                  vvvv

                                                                                                  vv

                                                                                                  v

                                                                                                  v

                                                                                                  xBBxP tVVV

                                                                                                  x V

                                                                                                  nvv 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                  bull For Orthogonal Complement

                                                                                                  and

                                                                                                  V

                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                  bull For Orthogonal Complement

                                                                                                  and

                                                                                                  bull Parseval Inequality

                                                                                                  V

                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                  2

                                                                                                  1

                                                                                                  22

                                                                                                  1

                                                                                                  22 aavxxxP

                                                                                                  n

                                                                                                  ii

                                                                                                  n

                                                                                                  iiV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  (Real) Unitary Matrices with

                                                                                                  bull Orthonormal Basis Matrix

                                                                                                  (So All of Above Applies)

                                                                                                  ddU IUU t

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  (Real) Unitary Matrices with

                                                                                                  bull Orthonormal Basis Matrix

                                                                                                  (So All of Above Applies)

                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                  ddU IUU t

                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                  i ii 2

                                                                                                  1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  (Real) Unitary Matrices with

                                                                                                  bull Orthonormal Basis Matrix

                                                                                                  (So All of Above Applies)

                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                  ddU IUU t

                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                  i ii 2

                                                                                                  1

                                                                                                  U

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Singular Value Decomposition (SVD)

                                                                                                  For a Matrix

                                                                                                  Find

                                                                                                  ndX

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Singular Value Decomposition (SVD)

                                                                                                  For a Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  with Entries

                                                                                                  called Singular Values

                                                                                                  ndX

                                                                                                  ndS

                                                                                                  )min(1 ndss

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Singular Value Decomposition (SVD)

                                                                                                  For a Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  with Entries

                                                                                                  called Singular Values

                                                                                                  And Unitary (Rotation) Matrices

                                                                                                  (recall )

                                                                                                  ndX

                                                                                                  ndS

                                                                                                  )min(1 ndss

                                                                                                  ddU nnV

                                                                                                  IVVUU tt

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Singular Value Decomposition (SVD)

                                                                                                  For a Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  with Entries

                                                                                                  called Singular Values

                                                                                                  And Unitary (Rotation) Matrices

                                                                                                  (recall )

                                                                                                  So That

                                                                                                  ndX

                                                                                                  ndS

                                                                                                  )min(1 ndss

                                                                                                  ddU nnV

                                                                                                  IVVUU tt tUSVX

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                  vVSUvVSUvX tt

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                  bull First rotate

                                                                                                  X

                                                                                                  vVSUvVSUvX tt

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                  bull First rotate

                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                  X

                                                                                                  vVSUvVSUvX tt

                                                                                                  is

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                  bull First rotate

                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                  bull Third rotate again

                                                                                                  X

                                                                                                  vVSUvVSUvX tt

                                                                                                  is

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                  bull First rotate

                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                  bull Third rotate again

                                                                                                  bull ie have diagonalized the transformation

                                                                                                  X

                                                                                                  vVSUvVSUvX tt

                                                                                                  is

                                                                                                  SVD Compact Representation

                                                                                                  Useful Labeling

                                                                                                  Singular Values in Increasing Order

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  )min(1 dnss

                                                                                                  SVD Compact Representation

                                                                                                  Useful Labeling

                                                                                                  Singular Values in Increasing Order

                                                                                                  Note singular values = 0 can be omitted

                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  )min(1 dnss

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  r

                                                                                                  SVD Compact Representation

                                                                                                  Useful Labeling

                                                                                                  Singular Values in Increasing Order

                                                                                                  Note singular values = 0 can be omitted

                                                                                                  Let = of positive singular values

                                                                                                  )min(1 dnss

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  r

                                                                                                  SVD Compact Representation

                                                                                                  Useful Labeling

                                                                                                  Singular Values in Increasing Order

                                                                                                  Note singular values = 0 can be omitted

                                                                                                  Let = of positive singular values

                                                                                                  Then

                                                                                                  Where are truncations of

                                                                                                  trnrrrd VSUX

                                                                                                  VSU

                                                                                                  )min(1 dnss

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Full Representation

                                                                                                  =

                                                                                                  Graphics Display Assumes

                                                                                                  ndX ddU ndS nn

                                                                                                  tV

                                                                                                  nd

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Full Representation

                                                                                                  =

                                                                                                  Full Rank Basis Matrix

                                                                                                  ndX ddU ndS nn

                                                                                                  tV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Full Representation

                                                                                                  =

                                                                                                  Full Rank Basis Matrix

                                                                                                  All 0s in Bottom

                                                                                                  ndX ddU ndS nn

                                                                                                  tV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Reduced Representation

                                                                                                  =

                                                                                                  These Columns Get 0ed Out

                                                                                                  ndX ddU nnS nn

                                                                                                  tV

                                                                                                  nnd 0

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Reduced Representation

                                                                                                  =ndX ndU nnS nn

                                                                                                  tV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Reduced Representation

                                                                                                  =

                                                                                                  Also Some of These May be 0

                                                                                                  ndX ndU nnS nn

                                                                                                  tV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Compact Representation

                                                                                                  = ndX rdU

                                                                                                  rrS nrtV

                                                                                                  0

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Compact Representation

                                                                                                  =

                                                                                                  These Get 0ed Out

                                                                                                  ndX rdU

                                                                                                  rrS nrtV

                                                                                                  0

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  SVD Compact Representation

                                                                                                  = ndX rdU

                                                                                                  rrS nrtV

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition

                                                                                                  For a (Symmetric) Square Matrix

                                                                                                  Find

                                                                                                  ddX

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition

                                                                                                  For a (Symmetric) Square Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  Called Eigenvalues

                                                                                                  Convenient Ordering

                                                                                                  ddX

                                                                                                  d

                                                                                                  D

                                                                                                  0

                                                                                                  01

                                                                                                  n 1

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition

                                                                                                  For a (Symmetric) Square Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  And an Orthonormal Matrix

                                                                                                  (ie )

                                                                                                  ddX

                                                                                                  d

                                                                                                  D

                                                                                                  0

                                                                                                  01

                                                                                                  ddB

                                                                                                  ddtt IBBBB

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition

                                                                                                  For a (Symmetric) Square Matrix

                                                                                                  Find a Diagonal Matrix

                                                                                                  And an Orthonormal Matrix

                                                                                                  (ie )

                                                                                                  So that ie

                                                                                                  ddX

                                                                                                  d

                                                                                                  D

                                                                                                  0

                                                                                                  01

                                                                                                  ddB

                                                                                                  ddtt IBBBB

                                                                                                  DBBX tBDBX

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                  (looks similar)

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                  Complex (uses )

                                                                                                  VU

                                                                                                  1i

                                                                                                  Review of Linear Algebra (Cont)

                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                  VU

                                                                                                  X

                                                                                                  BVU

                                                                                                  1i

                                                                                                  • Participant Presentations
                                                                                                  • PCA to find clusters
                                                                                                  • Statistical Smoothing
                                                                                                  • Kernel Density Estimation
                                                                                                  • Scatterplot Smoothing
                                                                                                  • Statistical Smoothing (2)
                                                                                                  • SiZer Background
                                                                                                  • SiZer Background (2)
                                                                                                  • SiZer Background (3)
                                                                                                  • SiZer Background (4)
                                                                                                  • SiZer Background (5)
                                                                                                  • SiZer Overview
                                                                                                  • PCA to find clusters (2)
                                                                                                  • PCA to find clusters (3)
                                                                                                  • PCA to find clusters (4)
                                                                                                  • PCA to find clusters (5)
                                                                                                  • PCA to find clusters (6)
                                                                                                  • PCA to find clusters (7)
                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                  • Frequency 2 Analysis
                                                                                                  • Frequency 2 Analysis (2)
                                                                                                  • Frequency 2 Analysis (3)
                                                                                                  • Yeast Cell Cycle
                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                  • Reclassification of Major Genes
                                                                                                  • Compare to Previous Classifrsquon
                                                                                                  • New Subpopulation View
                                                                                                  • New Subpopulation View (2)
                                                                                                  • Detailed Look at PCA
                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                  • An Interesting Historical Note
                                                                                                  • An Interesting Historical Note (2)
                                                                                                  • Detailed Look at PCA (2)
                                                                                                  • Detailed Look at PCA (3)
                                                                                                  • Review of Linear Algebra
                                                                                                  • Review of Linear Algebra (2)
                                                                                                  • Review of Linear Algebra (3)
                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Basis of Subspace Set of Vectors that

                                                                                                    bull Span ie Everything is a Lin Com of them

                                                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Basis of Subspace Set of Vectors that

                                                                                                    bull Span ie Everything is a Lin Com of them

                                                                                                    bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                    bull eg ldquoUnit Vector Basisrdquo

                                                                                                    bull Since

                                                                                                    d

                                                                                                    1

                                                                                                    0

                                                                                                    0

                                                                                                    0

                                                                                                    1

                                                                                                    0

                                                                                                    0

                                                                                                    0

                                                                                                    1

                                                                                                    1

                                                                                                    0

                                                                                                    0

                                                                                                    0

                                                                                                    1

                                                                                                    0

                                                                                                    0

                                                                                                    0

                                                                                                    1

                                                                                                    212

                                                                                                    1

                                                                                                    d

                                                                                                    d

                                                                                                    xxx

                                                                                                    x

                                                                                                    x

                                                                                                    x

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Basis Matrix of subspace of

                                                                                                    Given a basis

                                                                                                    create matrix of columns

                                                                                                    dnvv 1

                                                                                                    nddnd

                                                                                                    n

                                                                                                    n

                                                                                                    vv

                                                                                                    vv

                                                                                                    vvB

                                                                                                    1

                                                                                                    111

                                                                                                    1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                                    where

                                                                                                    n

                                                                                                    iii aBva

                                                                                                    1

                                                                                                    na

                                                                                                    a

                                                                                                    a 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                                    where

                                                                                                    Note Right Multiplication Gives

                                                                                                    Linear Combination of Column Vectors

                                                                                                    n

                                                                                                    iii aBva

                                                                                                    1

                                                                                                    na

                                                                                                    a

                                                                                                    a 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Then linear combo is a matrix multiplicatrsquon

                                                                                                    where

                                                                                                    Check sizes

                                                                                                    n

                                                                                                    iii aBva

                                                                                                    1

                                                                                                    na

                                                                                                    a

                                                                                                    a 1

                                                                                                    )1()(1 nndd

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                    For matrices

                                                                                                    Define the Matrix Product

                                                                                                    mkk

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    nmm

                                                                                                    n

                                                                                                    bb

                                                                                                    bb

                                                                                                    B

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iniik

                                                                                                    m

                                                                                                    iiik

                                                                                                    m

                                                                                                    inii

                                                                                                    m

                                                                                                    iii

                                                                                                    baba

                                                                                                    baba

                                                                                                    AB

                                                                                                    1

                                                                                                    11

                                                                                                    11

                                                                                                    111

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                    For matrices

                                                                                                    Define the Matrix Product

                                                                                                    (Inner Products of Rows With Columns )

                                                                                                    mkk

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    nmm

                                                                                                    n

                                                                                                    bb

                                                                                                    bb

                                                                                                    B

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iniik

                                                                                                    m

                                                                                                    iiik

                                                                                                    m

                                                                                                    inii

                                                                                                    m

                                                                                                    iii

                                                                                                    baba

                                                                                                    baba

                                                                                                    AB

                                                                                                    1

                                                                                                    11

                                                                                                    11

                                                                                                    111

                                                                                                    A B

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                    For matrices

                                                                                                    Define the Matrix Product

                                                                                                    (Inner Products of Rows With Columns )

                                                                                                    (Composition of Linear Transformations)

                                                                                                    mkk

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    nmm

                                                                                                    n

                                                                                                    bb

                                                                                                    bb

                                                                                                    B

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iniik

                                                                                                    m

                                                                                                    iiik

                                                                                                    m

                                                                                                    inii

                                                                                                    m

                                                                                                    iii

                                                                                                    baba

                                                                                                    baba

                                                                                                    AB

                                                                                                    1

                                                                                                    11

                                                                                                    11

                                                                                                    111

                                                                                                    A B

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                    For matrices

                                                                                                    Define the Matrix Product

                                                                                                    (Inner Products of Rows With Columns )

                                                                                                    (Composition of Linear Transformations)

                                                                                                    Often Useful to Check Sizes

                                                                                                    mkk

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    nmm

                                                                                                    n

                                                                                                    bb

                                                                                                    bb

                                                                                                    B

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iniik

                                                                                                    m

                                                                                                    iiik

                                                                                                    m

                                                                                                    inii

                                                                                                    m

                                                                                                    iii

                                                                                                    baba

                                                                                                    baba

                                                                                                    AB

                                                                                                    1

                                                                                                    11

                                                                                                    11

                                                                                                    111

                                                                                                    nmmknk

                                                                                                    A B

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                    For matrices

                                                                                                    Define the Matrix Product

                                                                                                    (Inner Products of Rows With Columns )

                                                                                                    (Composition of Linear Transformations)

                                                                                                    Often Useful to Check Sizes

                                                                                                    mkk

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    nmm

                                                                                                    n

                                                                                                    bb

                                                                                                    bb

                                                                                                    B

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iniik

                                                                                                    m

                                                                                                    iiik

                                                                                                    m

                                                                                                    inii

                                                                                                    m

                                                                                                    iii

                                                                                                    baba

                                                                                                    baba

                                                                                                    AB

                                                                                                    1

                                                                                                    11

                                                                                                    11

                                                                                                    111

                                                                                                    nmmknk

                                                                                                    A B

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Matrix Trace

                                                                                                    bull For a Square Matrix

                                                                                                    bull Define

                                                                                                    mmm

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iiiaAtr

                                                                                                    1)(

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Matrix Trace

                                                                                                    bull For a Square Matrix

                                                                                                    bull Define

                                                                                                    bull Trace Commutes with Matrix Multiplication

                                                                                                    mmm

                                                                                                    m

                                                                                                    aa

                                                                                                    aa

                                                                                                    A

                                                                                                    1

                                                                                                    111

                                                                                                    m

                                                                                                    iiiaAtr

                                                                                                    1)(

                                                                                                    BAtrABtr

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                    bull (Use Basis Above)

                                                                                                    bull eg dim of a line is 1

                                                                                                    bull eg dim of a plane is 2

                                                                                                    dd dim

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                    bull (Use Basis Above)

                                                                                                    bull eg dim of a line is 1

                                                                                                    bull eg dim of a plane is 2

                                                                                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                    (in Statistical Uses eg ANOVA)

                                                                                                    dd dim

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Norm of a Vector

                                                                                                    bull in d 21

                                                                                                    21

                                                                                                    1

                                                                                                    2 xxxx td

                                                                                                    jj

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Norm of a Vector

                                                                                                    bull in

                                                                                                    bull Idea length of the vector

                                                                                                    d 2121

                                                                                                    1

                                                                                                    2 xxxx td

                                                                                                    jj

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Norm of a Vector

                                                                                                    bull in

                                                                                                    bull Idea length of the vector

                                                                                                    bull Note strange properties for high

                                                                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                                                                    d 2121

                                                                                                    1

                                                                                                    2 xxxx td

                                                                                                    jj

                                                                                                    d

                                                                                                    d

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Norm of a Vector (cont)

                                                                                                    bull Length Normalized Vector

                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                    amp is a Direction Vector)

                                                                                                    x

                                                                                                    x

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Norm of a Vector (cont)

                                                                                                    bull Length Normalized Vector

                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                    amp is a Direction Vector)

                                                                                                    bull Define Distance as

                                                                                                    x

                                                                                                    x

                                                                                                    yxyxyxyxd t

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Inner (Dot Scalar) Product

                                                                                                    bull for Vectors and

                                                                                                    yxyxyx td

                                                                                                    jjj

                                                                                                    1

                                                                                                    x y

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Inner (Dot Scalar) Product

                                                                                                    bull for Vectors and

                                                                                                    bull Related to Norm via

                                                                                                    yxyxyx td

                                                                                                    jjj

                                                                                                    1

                                                                                                    x y

                                                                                                    21 xxx

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                    yyxx

                                                                                                    yx

                                                                                                    yx

                                                                                                    yxyxangle

                                                                                                    tt

                                                                                                    t

                                                                                                    11 cos

                                                                                                    cos

                                                                                                    x y

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                    if and only if

                                                                                                    yyxx

                                                                                                    yx

                                                                                                    yx

                                                                                                    yxyxangle

                                                                                                    tt

                                                                                                    t

                                                                                                    11 cos

                                                                                                    cos

                                                                                                    x y

                                                                                                    yx 0 yx

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis

                                                                                                    bull All Orthogonal to each other

                                                                                                    ie for

                                                                                                    bull All have Length 1

                                                                                                    ie for

                                                                                                    nvv 1

                                                                                                    1 ii vv

                                                                                                    0 ii vv ii

                                                                                                    ni 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis (cont)

                                                                                                    bull Spectral Representation

                                                                                                    where

                                                                                                    nvv 1

                                                                                                    n

                                                                                                    iii vax

                                                                                                    1

                                                                                                    ii vxa

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis (cont)

                                                                                                    bull Spectral Representation

                                                                                                    where

                                                                                                    (Coefficient is Inner Product Cool Notation)

                                                                                                    nvv 1

                                                                                                    n

                                                                                                    iii vax

                                                                                                    1

                                                                                                    ii vxa

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis (cont)

                                                                                                    bull Spectral Representation

                                                                                                    where

                                                                                                    Check

                                                                                                    nvv 1

                                                                                                    n

                                                                                                    iii vax

                                                                                                    1

                                                                                                    ii vxa

                                                                                                    iii

                                                                                                    n

                                                                                                    iii

                                                                                                    n

                                                                                                    iiii avvavvavx

                                                                                                    1

                                                                                                    1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis (cont)

                                                                                                    bull Spectral Representation

                                                                                                    where

                                                                                                    Check

                                                                                                    bull Matrix Notation where ie

                                                                                                    For the Basis Matrix

                                                                                                    nvv 1

                                                                                                    n

                                                                                                    iii vax

                                                                                                    1

                                                                                                    ii vxa

                                                                                                    iii

                                                                                                    n

                                                                                                    iii

                                                                                                    n

                                                                                                    iiii avvavvavx

                                                                                                    1

                                                                                                    1

                                                                                                    aBx Bxa tt xBa t

                                                                                                    nvvB 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Orthonormal Basis (cont)

                                                                                                    bull Spectral Representation

                                                                                                    where

                                                                                                    Check

                                                                                                    bull Matrix Notation where ie

                                                                                                    is called transform of

                                                                                                    (eg Fourier or Wavelet)

                                                                                                    nvv 1

                                                                                                    n

                                                                                                    iii vax

                                                                                                    1

                                                                                                    ii vxa

                                                                                                    iii

                                                                                                    n

                                                                                                    iii

                                                                                                    n

                                                                                                    iiii avvavvavx

                                                                                                    1

                                                                                                    1

                                                                                                    aBx Bxa tt xBa t

                                                                                                    xa

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Parseval identity for

                                                                                                    in subsp genrsquod by o n basis

                                                                                                    x

                                                                                                    nvv 1

                                                                                                    2

                                                                                                    1

                                                                                                    22

                                                                                                    1

                                                                                                    2 aavxx

                                                                                                    n

                                                                                                    ii

                                                                                                    n

                                                                                                    ii

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Parseval identity for

                                                                                                    in subsp genrsquod by o n basis

                                                                                                    bull Pythagorean theorem

                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                    bull ANOVA - sums of squares

                                                                                                    x

                                                                                                    nvv 1

                                                                                                    2

                                                                                                    1

                                                                                                    22

                                                                                                    1

                                                                                                    2 aavxx

                                                                                                    n

                                                                                                    ii

                                                                                                    n

                                                                                                    ii

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Parseval identity for

                                                                                                    in subsp genrsquod by o n basis

                                                                                                    bull Pythagorean theorem

                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                    bull ANOVA - sums of squares

                                                                                                    bull Transform has same length as

                                                                                                    ie ldquorotation in rdquo

                                                                                                    x

                                                                                                    nvv 1

                                                                                                    2

                                                                                                    1

                                                                                                    22

                                                                                                    1

                                                                                                    2 aavxx

                                                                                                    n

                                                                                                    ii

                                                                                                    n

                                                                                                    ii

                                                                                                    a xd

                                                                                                    Projection of a Vector onto a Subspace

                                                                                                    bull Idea Member of that is Closest to

                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                    xV

                                                                                                    V

                                                                                                    Projection of a Vector onto a Subspace

                                                                                                    bull Idea Member of that is Closest to

                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                    bull Find that Solves

                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                    xV

                                                                                                    V

                                                                                                    VxPV vxVv

                                                                                                    min

                                                                                                    Projection of a Vector onto a Subspace

                                                                                                    bull Idea Member of that is Closest to

                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                    bull Find that Solves

                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                    bull For Inner Product (Hilbert) Space

                                                                                                    Exists and is Unique

                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                    xV

                                                                                                    V

                                                                                                    VxPV vxVv

                                                                                                    min

                                                                                                    xPV

                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                    bull General Solution in for Basis Matrix

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    d VB

                                                                                                    xBBBBxP tVV

                                                                                                    tVVV

                                                                                                    1

                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                    bull General Solution in for Basis Matrix

                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                    (thus projection is another linear operation)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    d VB

                                                                                                    xBBBBxP tVV

                                                                                                    tVVV

                                                                                                    1

                                                                                                    tVV

                                                                                                    tVVV BBBBP

                                                                                                    1

                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                    bull General Solution in for Basis Matrix

                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                    (thus projection is another linear operation)

                                                                                                    (note same operation underlies least squares)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    d VB

                                                                                                    xBBBBxP tVV

                                                                                                    tVVV

                                                                                                    1

                                                                                                    tVV

                                                                                                    tVVV BBBBP

                                                                                                    1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Projection using Orthonormal Basis

                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                    nnVtV IBB

                                                                                                    10

                                                                                                    01

                                                                                                    1

                                                                                                    111

                                                                                                    1

                                                                                                    1

                                                                                                    nnn

                                                                                                    n

                                                                                                    ntn

                                                                                                    t

                                                                                                    vvvv

                                                                                                    vvvv

                                                                                                    vv

                                                                                                    v

                                                                                                    v

                                                                                                    nvv 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Projection using Orthonormal Basis

                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                    bull So =

                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                    (Recall Right Multrsquon)

                                                                                                    nnVtV IBB

                                                                                                    10

                                                                                                    01

                                                                                                    1

                                                                                                    111

                                                                                                    1

                                                                                                    1

                                                                                                    nnn

                                                                                                    n

                                                                                                    ntn

                                                                                                    t

                                                                                                    vvvv

                                                                                                    vvvv

                                                                                                    vv

                                                                                                    v

                                                                                                    v

                                                                                                    xBBxP tVVV

                                                                                                    x V

                                                                                                    nvv 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                    bull For Orthogonal Complement

                                                                                                    and

                                                                                                    V

                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                    bull For Orthogonal Complement

                                                                                                    and

                                                                                                    bull Parseval Inequality

                                                                                                    V

                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                    2

                                                                                                    1

                                                                                                    22

                                                                                                    1

                                                                                                    22 aavxxxP

                                                                                                    n

                                                                                                    ii

                                                                                                    n

                                                                                                    iiV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    (Real) Unitary Matrices with

                                                                                                    bull Orthonormal Basis Matrix

                                                                                                    (So All of Above Applies)

                                                                                                    ddU IUU t

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    (Real) Unitary Matrices with

                                                                                                    bull Orthonormal Basis Matrix

                                                                                                    (So All of Above Applies)

                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                    ddU IUU t

                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                    i ii 2

                                                                                                    1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    (Real) Unitary Matrices with

                                                                                                    bull Orthonormal Basis Matrix

                                                                                                    (So All of Above Applies)

                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                    ddU IUU t

                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                    i ii 2

                                                                                                    1

                                                                                                    U

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Singular Value Decomposition (SVD)

                                                                                                    For a Matrix

                                                                                                    Find

                                                                                                    ndX

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Singular Value Decomposition (SVD)

                                                                                                    For a Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    with Entries

                                                                                                    called Singular Values

                                                                                                    ndX

                                                                                                    ndS

                                                                                                    )min(1 ndss

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Singular Value Decomposition (SVD)

                                                                                                    For a Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    with Entries

                                                                                                    called Singular Values

                                                                                                    And Unitary (Rotation) Matrices

                                                                                                    (recall )

                                                                                                    ndX

                                                                                                    ndS

                                                                                                    )min(1 ndss

                                                                                                    ddU nnV

                                                                                                    IVVUU tt

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Singular Value Decomposition (SVD)

                                                                                                    For a Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    with Entries

                                                                                                    called Singular Values

                                                                                                    And Unitary (Rotation) Matrices

                                                                                                    (recall )

                                                                                                    So That

                                                                                                    ndX

                                                                                                    ndS

                                                                                                    )min(1 ndss

                                                                                                    ddU nnV

                                                                                                    IVVUU tt tUSVX

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                    vVSUvVSUvX tt

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                    bull First rotate

                                                                                                    X

                                                                                                    vVSUvVSUvX tt

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                    bull First rotate

                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                    X

                                                                                                    vVSUvVSUvX tt

                                                                                                    is

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                    bull First rotate

                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                    bull Third rotate again

                                                                                                    X

                                                                                                    vVSUvVSUvX tt

                                                                                                    is

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                    bull First rotate

                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                    bull Third rotate again

                                                                                                    bull ie have diagonalized the transformation

                                                                                                    X

                                                                                                    vVSUvVSUvX tt

                                                                                                    is

                                                                                                    SVD Compact Representation

                                                                                                    Useful Labeling

                                                                                                    Singular Values in Increasing Order

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    )min(1 dnss

                                                                                                    SVD Compact Representation

                                                                                                    Useful Labeling

                                                                                                    Singular Values in Increasing Order

                                                                                                    Note singular values = 0 can be omitted

                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    )min(1 dnss

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    r

                                                                                                    SVD Compact Representation

                                                                                                    Useful Labeling

                                                                                                    Singular Values in Increasing Order

                                                                                                    Note singular values = 0 can be omitted

                                                                                                    Let = of positive singular values

                                                                                                    )min(1 dnss

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    r

                                                                                                    SVD Compact Representation

                                                                                                    Useful Labeling

                                                                                                    Singular Values in Increasing Order

                                                                                                    Note singular values = 0 can be omitted

                                                                                                    Let = of positive singular values

                                                                                                    Then

                                                                                                    Where are truncations of

                                                                                                    trnrrrd VSUX

                                                                                                    VSU

                                                                                                    )min(1 dnss

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Full Representation

                                                                                                    =

                                                                                                    Graphics Display Assumes

                                                                                                    ndX ddU ndS nn

                                                                                                    tV

                                                                                                    nd

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Full Representation

                                                                                                    =

                                                                                                    Full Rank Basis Matrix

                                                                                                    ndX ddU ndS nn

                                                                                                    tV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Full Representation

                                                                                                    =

                                                                                                    Full Rank Basis Matrix

                                                                                                    All 0s in Bottom

                                                                                                    ndX ddU ndS nn

                                                                                                    tV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Reduced Representation

                                                                                                    =

                                                                                                    These Columns Get 0ed Out

                                                                                                    ndX ddU nnS nn

                                                                                                    tV

                                                                                                    nnd 0

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Reduced Representation

                                                                                                    =ndX ndU nnS nn

                                                                                                    tV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Reduced Representation

                                                                                                    =

                                                                                                    Also Some of These May be 0

                                                                                                    ndX ndU nnS nn

                                                                                                    tV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Compact Representation

                                                                                                    = ndX rdU

                                                                                                    rrS nrtV

                                                                                                    0

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Compact Representation

                                                                                                    =

                                                                                                    These Get 0ed Out

                                                                                                    ndX rdU

                                                                                                    rrS nrtV

                                                                                                    0

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    SVD Compact Representation

                                                                                                    = ndX rdU

                                                                                                    rrS nrtV

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition

                                                                                                    For a (Symmetric) Square Matrix

                                                                                                    Find

                                                                                                    ddX

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition

                                                                                                    For a (Symmetric) Square Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    Called Eigenvalues

                                                                                                    Convenient Ordering

                                                                                                    ddX

                                                                                                    d

                                                                                                    D

                                                                                                    0

                                                                                                    01

                                                                                                    n 1

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition

                                                                                                    For a (Symmetric) Square Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    And an Orthonormal Matrix

                                                                                                    (ie )

                                                                                                    ddX

                                                                                                    d

                                                                                                    D

                                                                                                    0

                                                                                                    01

                                                                                                    ddB

                                                                                                    ddtt IBBBB

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition

                                                                                                    For a (Symmetric) Square Matrix

                                                                                                    Find a Diagonal Matrix

                                                                                                    And an Orthonormal Matrix

                                                                                                    (ie )

                                                                                                    So that ie

                                                                                                    ddX

                                                                                                    d

                                                                                                    D

                                                                                                    0

                                                                                                    01

                                                                                                    ddB

                                                                                                    ddtt IBBBB

                                                                                                    DBBX tBDBX

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                    (looks similar)

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                    Complex (uses )

                                                                                                    VU

                                                                                                    1i

                                                                                                    Review of Linear Algebra (Cont)

                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                    VU

                                                                                                    X

                                                                                                    BVU

                                                                                                    1i

                                                                                                    • Participant Presentations
                                                                                                    • PCA to find clusters
                                                                                                    • Statistical Smoothing
                                                                                                    • Kernel Density Estimation
                                                                                                    • Scatterplot Smoothing
                                                                                                    • Statistical Smoothing (2)
                                                                                                    • SiZer Background
                                                                                                    • SiZer Background (2)
                                                                                                    • SiZer Background (3)
                                                                                                    • SiZer Background (4)
                                                                                                    • SiZer Background (5)
                                                                                                    • SiZer Overview
                                                                                                    • PCA to find clusters (2)
                                                                                                    • PCA to find clusters (3)
                                                                                                    • PCA to find clusters (4)
                                                                                                    • PCA to find clusters (5)
                                                                                                    • PCA to find clusters (6)
                                                                                                    • PCA to find clusters (7)
                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                    • Frequency 2 Analysis
                                                                                                    • Frequency 2 Analysis (2)
                                                                                                    • Frequency 2 Analysis (3)
                                                                                                    • Yeast Cell Cycle
                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                    • Reclassification of Major Genes
                                                                                                    • Compare to Previous Classifrsquon
                                                                                                    • New Subpopulation View
                                                                                                    • New Subpopulation View (2)
                                                                                                    • Detailed Look at PCA
                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                    • An Interesting Historical Note
                                                                                                    • An Interesting Historical Note (2)
                                                                                                    • Detailed Look at PCA (2)
                                                                                                    • Detailed Look at PCA (3)
                                                                                                    • Review of Linear Algebra
                                                                                                    • Review of Linear Algebra (2)
                                                                                                    • Review of Linear Algebra (3)
                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Basis of Subspace Set of Vectors that

                                                                                                      bull Span ie Everything is a Lin Com of them

                                                                                                      bull are Linearly Indeprsquot ie Lin Com is Unique

                                                                                                      bull eg ldquoUnit Vector Basisrdquo

                                                                                                      bull Since

                                                                                                      d

                                                                                                      1

                                                                                                      0

                                                                                                      0

                                                                                                      0

                                                                                                      1

                                                                                                      0

                                                                                                      0

                                                                                                      0

                                                                                                      1

                                                                                                      1

                                                                                                      0

                                                                                                      0

                                                                                                      0

                                                                                                      1

                                                                                                      0

                                                                                                      0

                                                                                                      0

                                                                                                      1

                                                                                                      212

                                                                                                      1

                                                                                                      d

                                                                                                      d

                                                                                                      xxx

                                                                                                      x

                                                                                                      x

                                                                                                      x

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Basis Matrix of subspace of

                                                                                                      Given a basis

                                                                                                      create matrix of columns

                                                                                                      dnvv 1

                                                                                                      nddnd

                                                                                                      n

                                                                                                      n

                                                                                                      vv

                                                                                                      vv

                                                                                                      vvB

                                                                                                      1

                                                                                                      111

                                                                                                      1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                                      where

                                                                                                      n

                                                                                                      iii aBva

                                                                                                      1

                                                                                                      na

                                                                                                      a

                                                                                                      a 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                                      where

                                                                                                      Note Right Multiplication Gives

                                                                                                      Linear Combination of Column Vectors

                                                                                                      n

                                                                                                      iii aBva

                                                                                                      1

                                                                                                      na

                                                                                                      a

                                                                                                      a 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Then linear combo is a matrix multiplicatrsquon

                                                                                                      where

                                                                                                      Check sizes

                                                                                                      n

                                                                                                      iii aBva

                                                                                                      1

                                                                                                      na

                                                                                                      a

                                                                                                      a 1

                                                                                                      )1()(1 nndd

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                      For matrices

                                                                                                      Define the Matrix Product

                                                                                                      mkk

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      nmm

                                                                                                      n

                                                                                                      bb

                                                                                                      bb

                                                                                                      B

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iniik

                                                                                                      m

                                                                                                      iiik

                                                                                                      m

                                                                                                      inii

                                                                                                      m

                                                                                                      iii

                                                                                                      baba

                                                                                                      baba

                                                                                                      AB

                                                                                                      1

                                                                                                      11

                                                                                                      11

                                                                                                      111

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                      For matrices

                                                                                                      Define the Matrix Product

                                                                                                      (Inner Products of Rows With Columns )

                                                                                                      mkk

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      nmm

                                                                                                      n

                                                                                                      bb

                                                                                                      bb

                                                                                                      B

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iniik

                                                                                                      m

                                                                                                      iiik

                                                                                                      m

                                                                                                      inii

                                                                                                      m

                                                                                                      iii

                                                                                                      baba

                                                                                                      baba

                                                                                                      AB

                                                                                                      1

                                                                                                      11

                                                                                                      11

                                                                                                      111

                                                                                                      A B

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                      For matrices

                                                                                                      Define the Matrix Product

                                                                                                      (Inner Products of Rows With Columns )

                                                                                                      (Composition of Linear Transformations)

                                                                                                      mkk

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      nmm

                                                                                                      n

                                                                                                      bb

                                                                                                      bb

                                                                                                      B

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iniik

                                                                                                      m

                                                                                                      iiik

                                                                                                      m

                                                                                                      inii

                                                                                                      m

                                                                                                      iii

                                                                                                      baba

                                                                                                      baba

                                                                                                      AB

                                                                                                      1

                                                                                                      11

                                                                                                      11

                                                                                                      111

                                                                                                      A B

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                      For matrices

                                                                                                      Define the Matrix Product

                                                                                                      (Inner Products of Rows With Columns )

                                                                                                      (Composition of Linear Transformations)

                                                                                                      Often Useful to Check Sizes

                                                                                                      mkk

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      nmm

                                                                                                      n

                                                                                                      bb

                                                                                                      bb

                                                                                                      B

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iniik

                                                                                                      m

                                                                                                      iiik

                                                                                                      m

                                                                                                      inii

                                                                                                      m

                                                                                                      iii

                                                                                                      baba

                                                                                                      baba

                                                                                                      AB

                                                                                                      1

                                                                                                      11

                                                                                                      11

                                                                                                      111

                                                                                                      nmmknk

                                                                                                      A B

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                      For matrices

                                                                                                      Define the Matrix Product

                                                                                                      (Inner Products of Rows With Columns )

                                                                                                      (Composition of Linear Transformations)

                                                                                                      Often Useful to Check Sizes

                                                                                                      mkk

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      nmm

                                                                                                      n

                                                                                                      bb

                                                                                                      bb

                                                                                                      B

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iniik

                                                                                                      m

                                                                                                      iiik

                                                                                                      m

                                                                                                      inii

                                                                                                      m

                                                                                                      iii

                                                                                                      baba

                                                                                                      baba

                                                                                                      AB

                                                                                                      1

                                                                                                      11

                                                                                                      11

                                                                                                      111

                                                                                                      nmmknk

                                                                                                      A B

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Matrix Trace

                                                                                                      bull For a Square Matrix

                                                                                                      bull Define

                                                                                                      mmm

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iiiaAtr

                                                                                                      1)(

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Matrix Trace

                                                                                                      bull For a Square Matrix

                                                                                                      bull Define

                                                                                                      bull Trace Commutes with Matrix Multiplication

                                                                                                      mmm

                                                                                                      m

                                                                                                      aa

                                                                                                      aa

                                                                                                      A

                                                                                                      1

                                                                                                      111

                                                                                                      m

                                                                                                      iiiaAtr

                                                                                                      1)(

                                                                                                      BAtrABtr

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                      bull (Use Basis Above)

                                                                                                      bull eg dim of a line is 1

                                                                                                      bull eg dim of a plane is 2

                                                                                                      dd dim

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                      bull (Use Basis Above)

                                                                                                      bull eg dim of a line is 1

                                                                                                      bull eg dim of a plane is 2

                                                                                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                      (in Statistical Uses eg ANOVA)

                                                                                                      dd dim

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Norm of a Vector

                                                                                                      bull in d 21

                                                                                                      21

                                                                                                      1

                                                                                                      2 xxxx td

                                                                                                      jj

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Norm of a Vector

                                                                                                      bull in

                                                                                                      bull Idea length of the vector

                                                                                                      d 2121

                                                                                                      1

                                                                                                      2 xxxx td

                                                                                                      jj

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Norm of a Vector

                                                                                                      bull in

                                                                                                      bull Idea length of the vector

                                                                                                      bull Note strange properties for high

                                                                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                                                                      d 2121

                                                                                                      1

                                                                                                      2 xxxx td

                                                                                                      jj

                                                                                                      d

                                                                                                      d

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Norm of a Vector (cont)

                                                                                                      bull Length Normalized Vector

                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                      amp is a Direction Vector)

                                                                                                      x

                                                                                                      x

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Norm of a Vector (cont)

                                                                                                      bull Length Normalized Vector

                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                      amp is a Direction Vector)

                                                                                                      bull Define Distance as

                                                                                                      x

                                                                                                      x

                                                                                                      yxyxyxyxd t

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Inner (Dot Scalar) Product

                                                                                                      bull for Vectors and

                                                                                                      yxyxyx td

                                                                                                      jjj

                                                                                                      1

                                                                                                      x y

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Inner (Dot Scalar) Product

                                                                                                      bull for Vectors and

                                                                                                      bull Related to Norm via

                                                                                                      yxyxyx td

                                                                                                      jjj

                                                                                                      1

                                                                                                      x y

                                                                                                      21 xxx

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                      yyxx

                                                                                                      yx

                                                                                                      yx

                                                                                                      yxyxangle

                                                                                                      tt

                                                                                                      t

                                                                                                      11 cos

                                                                                                      cos

                                                                                                      x y

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                      if and only if

                                                                                                      yyxx

                                                                                                      yx

                                                                                                      yx

                                                                                                      yxyxangle

                                                                                                      tt

                                                                                                      t

                                                                                                      11 cos

                                                                                                      cos

                                                                                                      x y

                                                                                                      yx 0 yx

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis

                                                                                                      bull All Orthogonal to each other

                                                                                                      ie for

                                                                                                      bull All have Length 1

                                                                                                      ie for

                                                                                                      nvv 1

                                                                                                      1 ii vv

                                                                                                      0 ii vv ii

                                                                                                      ni 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis (cont)

                                                                                                      bull Spectral Representation

                                                                                                      where

                                                                                                      nvv 1

                                                                                                      n

                                                                                                      iii vax

                                                                                                      1

                                                                                                      ii vxa

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis (cont)

                                                                                                      bull Spectral Representation

                                                                                                      where

                                                                                                      (Coefficient is Inner Product Cool Notation)

                                                                                                      nvv 1

                                                                                                      n

                                                                                                      iii vax

                                                                                                      1

                                                                                                      ii vxa

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis (cont)

                                                                                                      bull Spectral Representation

                                                                                                      where

                                                                                                      Check

                                                                                                      nvv 1

                                                                                                      n

                                                                                                      iii vax

                                                                                                      1

                                                                                                      ii vxa

                                                                                                      iii

                                                                                                      n

                                                                                                      iii

                                                                                                      n

                                                                                                      iiii avvavvavx

                                                                                                      1

                                                                                                      1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis (cont)

                                                                                                      bull Spectral Representation

                                                                                                      where

                                                                                                      Check

                                                                                                      bull Matrix Notation where ie

                                                                                                      For the Basis Matrix

                                                                                                      nvv 1

                                                                                                      n

                                                                                                      iii vax

                                                                                                      1

                                                                                                      ii vxa

                                                                                                      iii

                                                                                                      n

                                                                                                      iii

                                                                                                      n

                                                                                                      iiii avvavvavx

                                                                                                      1

                                                                                                      1

                                                                                                      aBx Bxa tt xBa t

                                                                                                      nvvB 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Orthonormal Basis (cont)

                                                                                                      bull Spectral Representation

                                                                                                      where

                                                                                                      Check

                                                                                                      bull Matrix Notation where ie

                                                                                                      is called transform of

                                                                                                      (eg Fourier or Wavelet)

                                                                                                      nvv 1

                                                                                                      n

                                                                                                      iii vax

                                                                                                      1

                                                                                                      ii vxa

                                                                                                      iii

                                                                                                      n

                                                                                                      iii

                                                                                                      n

                                                                                                      iiii avvavvavx

                                                                                                      1

                                                                                                      1

                                                                                                      aBx Bxa tt xBa t

                                                                                                      xa

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Parseval identity for

                                                                                                      in subsp genrsquod by o n basis

                                                                                                      x

                                                                                                      nvv 1

                                                                                                      2

                                                                                                      1

                                                                                                      22

                                                                                                      1

                                                                                                      2 aavxx

                                                                                                      n

                                                                                                      ii

                                                                                                      n

                                                                                                      ii

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Parseval identity for

                                                                                                      in subsp genrsquod by o n basis

                                                                                                      bull Pythagorean theorem

                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                      bull ANOVA - sums of squares

                                                                                                      x

                                                                                                      nvv 1

                                                                                                      2

                                                                                                      1

                                                                                                      22

                                                                                                      1

                                                                                                      2 aavxx

                                                                                                      n

                                                                                                      ii

                                                                                                      n

                                                                                                      ii

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Parseval identity for

                                                                                                      in subsp genrsquod by o n basis

                                                                                                      bull Pythagorean theorem

                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                      bull ANOVA - sums of squares

                                                                                                      bull Transform has same length as

                                                                                                      ie ldquorotation in rdquo

                                                                                                      x

                                                                                                      nvv 1

                                                                                                      2

                                                                                                      1

                                                                                                      22

                                                                                                      1

                                                                                                      2 aavxx

                                                                                                      n

                                                                                                      ii

                                                                                                      n

                                                                                                      ii

                                                                                                      a xd

                                                                                                      Projection of a Vector onto a Subspace

                                                                                                      bull Idea Member of that is Closest to

                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                      xV

                                                                                                      V

                                                                                                      Projection of a Vector onto a Subspace

                                                                                                      bull Idea Member of that is Closest to

                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                      bull Find that Solves

                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                      xV

                                                                                                      V

                                                                                                      VxPV vxVv

                                                                                                      min

                                                                                                      Projection of a Vector onto a Subspace

                                                                                                      bull Idea Member of that is Closest to

                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                      bull Find that Solves

                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                      bull For Inner Product (Hilbert) Space

                                                                                                      Exists and is Unique

                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                      xV

                                                                                                      V

                                                                                                      VxPV vxVv

                                                                                                      min

                                                                                                      xPV

                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                      bull General Solution in for Basis Matrix

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      d VB

                                                                                                      xBBBBxP tVV

                                                                                                      tVVV

                                                                                                      1

                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                      bull General Solution in for Basis Matrix

                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                      (thus projection is another linear operation)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      d VB

                                                                                                      xBBBBxP tVV

                                                                                                      tVVV

                                                                                                      1

                                                                                                      tVV

                                                                                                      tVVV BBBBP

                                                                                                      1

                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                      bull General Solution in for Basis Matrix

                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                      (thus projection is another linear operation)

                                                                                                      (note same operation underlies least squares)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      d VB

                                                                                                      xBBBBxP tVV

                                                                                                      tVVV

                                                                                                      1

                                                                                                      tVV

                                                                                                      tVVV BBBBP

                                                                                                      1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Projection using Orthonormal Basis

                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                      nnVtV IBB

                                                                                                      10

                                                                                                      01

                                                                                                      1

                                                                                                      111

                                                                                                      1

                                                                                                      1

                                                                                                      nnn

                                                                                                      n

                                                                                                      ntn

                                                                                                      t

                                                                                                      vvvv

                                                                                                      vvvv

                                                                                                      vv

                                                                                                      v

                                                                                                      v

                                                                                                      nvv 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Projection using Orthonormal Basis

                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                      bull So =

                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                      (Recall Right Multrsquon)

                                                                                                      nnVtV IBB

                                                                                                      10

                                                                                                      01

                                                                                                      1

                                                                                                      111

                                                                                                      1

                                                                                                      1

                                                                                                      nnn

                                                                                                      n

                                                                                                      ntn

                                                                                                      t

                                                                                                      vvvv

                                                                                                      vvvv

                                                                                                      vv

                                                                                                      v

                                                                                                      v

                                                                                                      xBBxP tVVV

                                                                                                      x V

                                                                                                      nvv 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                      bull For Orthogonal Complement

                                                                                                      and

                                                                                                      V

                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                      bull For Orthogonal Complement

                                                                                                      and

                                                                                                      bull Parseval Inequality

                                                                                                      V

                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                      2

                                                                                                      1

                                                                                                      22

                                                                                                      1

                                                                                                      22 aavxxxP

                                                                                                      n

                                                                                                      ii

                                                                                                      n

                                                                                                      iiV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      (Real) Unitary Matrices with

                                                                                                      bull Orthonormal Basis Matrix

                                                                                                      (So All of Above Applies)

                                                                                                      ddU IUU t

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      (Real) Unitary Matrices with

                                                                                                      bull Orthonormal Basis Matrix

                                                                                                      (So All of Above Applies)

                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                      ddU IUU t

                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                      i ii 2

                                                                                                      1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      (Real) Unitary Matrices with

                                                                                                      bull Orthonormal Basis Matrix

                                                                                                      (So All of Above Applies)

                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                      ddU IUU t

                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                      i ii 2

                                                                                                      1

                                                                                                      U

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Singular Value Decomposition (SVD)

                                                                                                      For a Matrix

                                                                                                      Find

                                                                                                      ndX

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Singular Value Decomposition (SVD)

                                                                                                      For a Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      with Entries

                                                                                                      called Singular Values

                                                                                                      ndX

                                                                                                      ndS

                                                                                                      )min(1 ndss

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Singular Value Decomposition (SVD)

                                                                                                      For a Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      with Entries

                                                                                                      called Singular Values

                                                                                                      And Unitary (Rotation) Matrices

                                                                                                      (recall )

                                                                                                      ndX

                                                                                                      ndS

                                                                                                      )min(1 ndss

                                                                                                      ddU nnV

                                                                                                      IVVUU tt

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Singular Value Decomposition (SVD)

                                                                                                      For a Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      with Entries

                                                                                                      called Singular Values

                                                                                                      And Unitary (Rotation) Matrices

                                                                                                      (recall )

                                                                                                      So That

                                                                                                      ndX

                                                                                                      ndS

                                                                                                      )min(1 ndss

                                                                                                      ddU nnV

                                                                                                      IVVUU tt tUSVX

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                      vVSUvVSUvX tt

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                      bull First rotate

                                                                                                      X

                                                                                                      vVSUvVSUvX tt

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                      bull First rotate

                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                      X

                                                                                                      vVSUvVSUvX tt

                                                                                                      is

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                      bull First rotate

                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                      bull Third rotate again

                                                                                                      X

                                                                                                      vVSUvVSUvX tt

                                                                                                      is

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                      bull First rotate

                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                      bull Third rotate again

                                                                                                      bull ie have diagonalized the transformation

                                                                                                      X

                                                                                                      vVSUvVSUvX tt

                                                                                                      is

                                                                                                      SVD Compact Representation

                                                                                                      Useful Labeling

                                                                                                      Singular Values in Increasing Order

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      )min(1 dnss

                                                                                                      SVD Compact Representation

                                                                                                      Useful Labeling

                                                                                                      Singular Values in Increasing Order

                                                                                                      Note singular values = 0 can be omitted

                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      )min(1 dnss

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      r

                                                                                                      SVD Compact Representation

                                                                                                      Useful Labeling

                                                                                                      Singular Values in Increasing Order

                                                                                                      Note singular values = 0 can be omitted

                                                                                                      Let = of positive singular values

                                                                                                      )min(1 dnss

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      r

                                                                                                      SVD Compact Representation

                                                                                                      Useful Labeling

                                                                                                      Singular Values in Increasing Order

                                                                                                      Note singular values = 0 can be omitted

                                                                                                      Let = of positive singular values

                                                                                                      Then

                                                                                                      Where are truncations of

                                                                                                      trnrrrd VSUX

                                                                                                      VSU

                                                                                                      )min(1 dnss

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Full Representation

                                                                                                      =

                                                                                                      Graphics Display Assumes

                                                                                                      ndX ddU ndS nn

                                                                                                      tV

                                                                                                      nd

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Full Representation

                                                                                                      =

                                                                                                      Full Rank Basis Matrix

                                                                                                      ndX ddU ndS nn

                                                                                                      tV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Full Representation

                                                                                                      =

                                                                                                      Full Rank Basis Matrix

                                                                                                      All 0s in Bottom

                                                                                                      ndX ddU ndS nn

                                                                                                      tV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Reduced Representation

                                                                                                      =

                                                                                                      These Columns Get 0ed Out

                                                                                                      ndX ddU nnS nn

                                                                                                      tV

                                                                                                      nnd 0

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Reduced Representation

                                                                                                      =ndX ndU nnS nn

                                                                                                      tV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Reduced Representation

                                                                                                      =

                                                                                                      Also Some of These May be 0

                                                                                                      ndX ndU nnS nn

                                                                                                      tV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Compact Representation

                                                                                                      = ndX rdU

                                                                                                      rrS nrtV

                                                                                                      0

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Compact Representation

                                                                                                      =

                                                                                                      These Get 0ed Out

                                                                                                      ndX rdU

                                                                                                      rrS nrtV

                                                                                                      0

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      SVD Compact Representation

                                                                                                      = ndX rdU

                                                                                                      rrS nrtV

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition

                                                                                                      For a (Symmetric) Square Matrix

                                                                                                      Find

                                                                                                      ddX

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition

                                                                                                      For a (Symmetric) Square Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      Called Eigenvalues

                                                                                                      Convenient Ordering

                                                                                                      ddX

                                                                                                      d

                                                                                                      D

                                                                                                      0

                                                                                                      01

                                                                                                      n 1

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition

                                                                                                      For a (Symmetric) Square Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      And an Orthonormal Matrix

                                                                                                      (ie )

                                                                                                      ddX

                                                                                                      d

                                                                                                      D

                                                                                                      0

                                                                                                      01

                                                                                                      ddB

                                                                                                      ddtt IBBBB

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition

                                                                                                      For a (Symmetric) Square Matrix

                                                                                                      Find a Diagonal Matrix

                                                                                                      And an Orthonormal Matrix

                                                                                                      (ie )

                                                                                                      So that ie

                                                                                                      ddX

                                                                                                      d

                                                                                                      D

                                                                                                      0

                                                                                                      01

                                                                                                      ddB

                                                                                                      ddtt IBBBB

                                                                                                      DBBX tBDBX

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                      (looks similar)

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                      Complex (uses )

                                                                                                      VU

                                                                                                      1i

                                                                                                      Review of Linear Algebra (Cont)

                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                      VU

                                                                                                      X

                                                                                                      BVU

                                                                                                      1i

                                                                                                      • Participant Presentations
                                                                                                      • PCA to find clusters
                                                                                                      • Statistical Smoothing
                                                                                                      • Kernel Density Estimation
                                                                                                      • Scatterplot Smoothing
                                                                                                      • Statistical Smoothing (2)
                                                                                                      • SiZer Background
                                                                                                      • SiZer Background (2)
                                                                                                      • SiZer Background (3)
                                                                                                      • SiZer Background (4)
                                                                                                      • SiZer Background (5)
                                                                                                      • SiZer Overview
                                                                                                      • PCA to find clusters (2)
                                                                                                      • PCA to find clusters (3)
                                                                                                      • PCA to find clusters (4)
                                                                                                      • PCA to find clusters (5)
                                                                                                      • PCA to find clusters (6)
                                                                                                      • PCA to find clusters (7)
                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                      • Frequency 2 Analysis
                                                                                                      • Frequency 2 Analysis (2)
                                                                                                      • Frequency 2 Analysis (3)
                                                                                                      • Yeast Cell Cycle
                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                      • Reclassification of Major Genes
                                                                                                      • Compare to Previous Classifrsquon
                                                                                                      • New Subpopulation View
                                                                                                      • New Subpopulation View (2)
                                                                                                      • Detailed Look at PCA
                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                      • An Interesting Historical Note
                                                                                                      • An Interesting Historical Note (2)
                                                                                                      • Detailed Look at PCA (2)
                                                                                                      • Detailed Look at PCA (3)
                                                                                                      • Review of Linear Algebra
                                                                                                      • Review of Linear Algebra (2)
                                                                                                      • Review of Linear Algebra (3)
                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Basis Matrix of subspace of

                                                                                                        Given a basis

                                                                                                        create matrix of columns

                                                                                                        dnvv 1

                                                                                                        nddnd

                                                                                                        n

                                                                                                        n

                                                                                                        vv

                                                                                                        vv

                                                                                                        vvB

                                                                                                        1

                                                                                                        111

                                                                                                        1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                                        where

                                                                                                        n

                                                                                                        iii aBva

                                                                                                        1

                                                                                                        na

                                                                                                        a

                                                                                                        a 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                                        where

                                                                                                        Note Right Multiplication Gives

                                                                                                        Linear Combination of Column Vectors

                                                                                                        n

                                                                                                        iii aBva

                                                                                                        1

                                                                                                        na

                                                                                                        a

                                                                                                        a 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Then linear combo is a matrix multiplicatrsquon

                                                                                                        where

                                                                                                        Check sizes

                                                                                                        n

                                                                                                        iii aBva

                                                                                                        1

                                                                                                        na

                                                                                                        a

                                                                                                        a 1

                                                                                                        )1()(1 nndd

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                        For matrices

                                                                                                        Define the Matrix Product

                                                                                                        mkk

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        nmm

                                                                                                        n

                                                                                                        bb

                                                                                                        bb

                                                                                                        B

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iniik

                                                                                                        m

                                                                                                        iiik

                                                                                                        m

                                                                                                        inii

                                                                                                        m

                                                                                                        iii

                                                                                                        baba

                                                                                                        baba

                                                                                                        AB

                                                                                                        1

                                                                                                        11

                                                                                                        11

                                                                                                        111

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                        For matrices

                                                                                                        Define the Matrix Product

                                                                                                        (Inner Products of Rows With Columns )

                                                                                                        mkk

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        nmm

                                                                                                        n

                                                                                                        bb

                                                                                                        bb

                                                                                                        B

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iniik

                                                                                                        m

                                                                                                        iiik

                                                                                                        m

                                                                                                        inii

                                                                                                        m

                                                                                                        iii

                                                                                                        baba

                                                                                                        baba

                                                                                                        AB

                                                                                                        1

                                                                                                        11

                                                                                                        11

                                                                                                        111

                                                                                                        A B

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                        For matrices

                                                                                                        Define the Matrix Product

                                                                                                        (Inner Products of Rows With Columns )

                                                                                                        (Composition of Linear Transformations)

                                                                                                        mkk

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        nmm

                                                                                                        n

                                                                                                        bb

                                                                                                        bb

                                                                                                        B

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iniik

                                                                                                        m

                                                                                                        iiik

                                                                                                        m

                                                                                                        inii

                                                                                                        m

                                                                                                        iii

                                                                                                        baba

                                                                                                        baba

                                                                                                        AB

                                                                                                        1

                                                                                                        11

                                                                                                        11

                                                                                                        111

                                                                                                        A B

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                        For matrices

                                                                                                        Define the Matrix Product

                                                                                                        (Inner Products of Rows With Columns )

                                                                                                        (Composition of Linear Transformations)

                                                                                                        Often Useful to Check Sizes

                                                                                                        mkk

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        nmm

                                                                                                        n

                                                                                                        bb

                                                                                                        bb

                                                                                                        B

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iniik

                                                                                                        m

                                                                                                        iiik

                                                                                                        m

                                                                                                        inii

                                                                                                        m

                                                                                                        iii

                                                                                                        baba

                                                                                                        baba

                                                                                                        AB

                                                                                                        1

                                                                                                        11

                                                                                                        11

                                                                                                        111

                                                                                                        nmmknk

                                                                                                        A B

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                        For matrices

                                                                                                        Define the Matrix Product

                                                                                                        (Inner Products of Rows With Columns )

                                                                                                        (Composition of Linear Transformations)

                                                                                                        Often Useful to Check Sizes

                                                                                                        mkk

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        nmm

                                                                                                        n

                                                                                                        bb

                                                                                                        bb

                                                                                                        B

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iniik

                                                                                                        m

                                                                                                        iiik

                                                                                                        m

                                                                                                        inii

                                                                                                        m

                                                                                                        iii

                                                                                                        baba

                                                                                                        baba

                                                                                                        AB

                                                                                                        1

                                                                                                        11

                                                                                                        11

                                                                                                        111

                                                                                                        nmmknk

                                                                                                        A B

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Matrix Trace

                                                                                                        bull For a Square Matrix

                                                                                                        bull Define

                                                                                                        mmm

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iiiaAtr

                                                                                                        1)(

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Matrix Trace

                                                                                                        bull For a Square Matrix

                                                                                                        bull Define

                                                                                                        bull Trace Commutes with Matrix Multiplication

                                                                                                        mmm

                                                                                                        m

                                                                                                        aa

                                                                                                        aa

                                                                                                        A

                                                                                                        1

                                                                                                        111

                                                                                                        m

                                                                                                        iiiaAtr

                                                                                                        1)(

                                                                                                        BAtrABtr

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                        bull (Use Basis Above)

                                                                                                        bull eg dim of a line is 1

                                                                                                        bull eg dim of a plane is 2

                                                                                                        dd dim

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                        bull (Use Basis Above)

                                                                                                        bull eg dim of a line is 1

                                                                                                        bull eg dim of a plane is 2

                                                                                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                        (in Statistical Uses eg ANOVA)

                                                                                                        dd dim

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Norm of a Vector

                                                                                                        bull in d 21

                                                                                                        21

                                                                                                        1

                                                                                                        2 xxxx td

                                                                                                        jj

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Norm of a Vector

                                                                                                        bull in

                                                                                                        bull Idea length of the vector

                                                                                                        d 2121

                                                                                                        1

                                                                                                        2 xxxx td

                                                                                                        jj

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Norm of a Vector

                                                                                                        bull in

                                                                                                        bull Idea length of the vector

                                                                                                        bull Note strange properties for high

                                                                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                                                                        d 2121

                                                                                                        1

                                                                                                        2 xxxx td

                                                                                                        jj

                                                                                                        d

                                                                                                        d

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Norm of a Vector (cont)

                                                                                                        bull Length Normalized Vector

                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                        amp is a Direction Vector)

                                                                                                        x

                                                                                                        x

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Norm of a Vector (cont)

                                                                                                        bull Length Normalized Vector

                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                        amp is a Direction Vector)

                                                                                                        bull Define Distance as

                                                                                                        x

                                                                                                        x

                                                                                                        yxyxyxyxd t

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Inner (Dot Scalar) Product

                                                                                                        bull for Vectors and

                                                                                                        yxyxyx td

                                                                                                        jjj

                                                                                                        1

                                                                                                        x y

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Inner (Dot Scalar) Product

                                                                                                        bull for Vectors and

                                                                                                        bull Related to Norm via

                                                                                                        yxyxyx td

                                                                                                        jjj

                                                                                                        1

                                                                                                        x y

                                                                                                        21 xxx

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                        yyxx

                                                                                                        yx

                                                                                                        yx

                                                                                                        yxyxangle

                                                                                                        tt

                                                                                                        t

                                                                                                        11 cos

                                                                                                        cos

                                                                                                        x y

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                        if and only if

                                                                                                        yyxx

                                                                                                        yx

                                                                                                        yx

                                                                                                        yxyxangle

                                                                                                        tt

                                                                                                        t

                                                                                                        11 cos

                                                                                                        cos

                                                                                                        x y

                                                                                                        yx 0 yx

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis

                                                                                                        bull All Orthogonal to each other

                                                                                                        ie for

                                                                                                        bull All have Length 1

                                                                                                        ie for

                                                                                                        nvv 1

                                                                                                        1 ii vv

                                                                                                        0 ii vv ii

                                                                                                        ni 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis (cont)

                                                                                                        bull Spectral Representation

                                                                                                        where

                                                                                                        nvv 1

                                                                                                        n

                                                                                                        iii vax

                                                                                                        1

                                                                                                        ii vxa

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis (cont)

                                                                                                        bull Spectral Representation

                                                                                                        where

                                                                                                        (Coefficient is Inner Product Cool Notation)

                                                                                                        nvv 1

                                                                                                        n

                                                                                                        iii vax

                                                                                                        1

                                                                                                        ii vxa

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis (cont)

                                                                                                        bull Spectral Representation

                                                                                                        where

                                                                                                        Check

                                                                                                        nvv 1

                                                                                                        n

                                                                                                        iii vax

                                                                                                        1

                                                                                                        ii vxa

                                                                                                        iii

                                                                                                        n

                                                                                                        iii

                                                                                                        n

                                                                                                        iiii avvavvavx

                                                                                                        1

                                                                                                        1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis (cont)

                                                                                                        bull Spectral Representation

                                                                                                        where

                                                                                                        Check

                                                                                                        bull Matrix Notation where ie

                                                                                                        For the Basis Matrix

                                                                                                        nvv 1

                                                                                                        n

                                                                                                        iii vax

                                                                                                        1

                                                                                                        ii vxa

                                                                                                        iii

                                                                                                        n

                                                                                                        iii

                                                                                                        n

                                                                                                        iiii avvavvavx

                                                                                                        1

                                                                                                        1

                                                                                                        aBx Bxa tt xBa t

                                                                                                        nvvB 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Orthonormal Basis (cont)

                                                                                                        bull Spectral Representation

                                                                                                        where

                                                                                                        Check

                                                                                                        bull Matrix Notation where ie

                                                                                                        is called transform of

                                                                                                        (eg Fourier or Wavelet)

                                                                                                        nvv 1

                                                                                                        n

                                                                                                        iii vax

                                                                                                        1

                                                                                                        ii vxa

                                                                                                        iii

                                                                                                        n

                                                                                                        iii

                                                                                                        n

                                                                                                        iiii avvavvavx

                                                                                                        1

                                                                                                        1

                                                                                                        aBx Bxa tt xBa t

                                                                                                        xa

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Parseval identity for

                                                                                                        in subsp genrsquod by o n basis

                                                                                                        x

                                                                                                        nvv 1

                                                                                                        2

                                                                                                        1

                                                                                                        22

                                                                                                        1

                                                                                                        2 aavxx

                                                                                                        n

                                                                                                        ii

                                                                                                        n

                                                                                                        ii

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Parseval identity for

                                                                                                        in subsp genrsquod by o n basis

                                                                                                        bull Pythagorean theorem

                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                        bull ANOVA - sums of squares

                                                                                                        x

                                                                                                        nvv 1

                                                                                                        2

                                                                                                        1

                                                                                                        22

                                                                                                        1

                                                                                                        2 aavxx

                                                                                                        n

                                                                                                        ii

                                                                                                        n

                                                                                                        ii

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Parseval identity for

                                                                                                        in subsp genrsquod by o n basis

                                                                                                        bull Pythagorean theorem

                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                        bull ANOVA - sums of squares

                                                                                                        bull Transform has same length as

                                                                                                        ie ldquorotation in rdquo

                                                                                                        x

                                                                                                        nvv 1

                                                                                                        2

                                                                                                        1

                                                                                                        22

                                                                                                        1

                                                                                                        2 aavxx

                                                                                                        n

                                                                                                        ii

                                                                                                        n

                                                                                                        ii

                                                                                                        a xd

                                                                                                        Projection of a Vector onto a Subspace

                                                                                                        bull Idea Member of that is Closest to

                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                        xV

                                                                                                        V

                                                                                                        Projection of a Vector onto a Subspace

                                                                                                        bull Idea Member of that is Closest to

                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                        bull Find that Solves

                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                        xV

                                                                                                        V

                                                                                                        VxPV vxVv

                                                                                                        min

                                                                                                        Projection of a Vector onto a Subspace

                                                                                                        bull Idea Member of that is Closest to

                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                        bull Find that Solves

                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                        bull For Inner Product (Hilbert) Space

                                                                                                        Exists and is Unique

                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                        xV

                                                                                                        V

                                                                                                        VxPV vxVv

                                                                                                        min

                                                                                                        xPV

                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                        bull General Solution in for Basis Matrix

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        d VB

                                                                                                        xBBBBxP tVV

                                                                                                        tVVV

                                                                                                        1

                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                        bull General Solution in for Basis Matrix

                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                        (thus projection is another linear operation)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        d VB

                                                                                                        xBBBBxP tVV

                                                                                                        tVVV

                                                                                                        1

                                                                                                        tVV

                                                                                                        tVVV BBBBP

                                                                                                        1

                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                        bull General Solution in for Basis Matrix

                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                        (thus projection is another linear operation)

                                                                                                        (note same operation underlies least squares)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        d VB

                                                                                                        xBBBBxP tVV

                                                                                                        tVVV

                                                                                                        1

                                                                                                        tVV

                                                                                                        tVVV BBBBP

                                                                                                        1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Projection using Orthonormal Basis

                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                        nnVtV IBB

                                                                                                        10

                                                                                                        01

                                                                                                        1

                                                                                                        111

                                                                                                        1

                                                                                                        1

                                                                                                        nnn

                                                                                                        n

                                                                                                        ntn

                                                                                                        t

                                                                                                        vvvv

                                                                                                        vvvv

                                                                                                        vv

                                                                                                        v

                                                                                                        v

                                                                                                        nvv 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Projection using Orthonormal Basis

                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                        bull So =

                                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                        (Recall Right Multrsquon)

                                                                                                        nnVtV IBB

                                                                                                        10

                                                                                                        01

                                                                                                        1

                                                                                                        111

                                                                                                        1

                                                                                                        1

                                                                                                        nnn

                                                                                                        n

                                                                                                        ntn

                                                                                                        t

                                                                                                        vvvv

                                                                                                        vvvv

                                                                                                        vv

                                                                                                        v

                                                                                                        v

                                                                                                        xBBxP tVVV

                                                                                                        x V

                                                                                                        nvv 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                        bull For Orthogonal Complement

                                                                                                        and

                                                                                                        V

                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                        bull For Orthogonal Complement

                                                                                                        and

                                                                                                        bull Parseval Inequality

                                                                                                        V

                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                        2

                                                                                                        1

                                                                                                        22

                                                                                                        1

                                                                                                        22 aavxxxP

                                                                                                        n

                                                                                                        ii

                                                                                                        n

                                                                                                        iiV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        (Real) Unitary Matrices with

                                                                                                        bull Orthonormal Basis Matrix

                                                                                                        (So All of Above Applies)

                                                                                                        ddU IUU t

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        (Real) Unitary Matrices with

                                                                                                        bull Orthonormal Basis Matrix

                                                                                                        (So All of Above Applies)

                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                        ddU IUU t

                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                        i ii 2

                                                                                                        1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        (Real) Unitary Matrices with

                                                                                                        bull Orthonormal Basis Matrix

                                                                                                        (So All of Above Applies)

                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                        ddU IUU t

                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                        i ii 2

                                                                                                        1

                                                                                                        U

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Singular Value Decomposition (SVD)

                                                                                                        For a Matrix

                                                                                                        Find

                                                                                                        ndX

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Singular Value Decomposition (SVD)

                                                                                                        For a Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        with Entries

                                                                                                        called Singular Values

                                                                                                        ndX

                                                                                                        ndS

                                                                                                        )min(1 ndss

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Singular Value Decomposition (SVD)

                                                                                                        For a Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        with Entries

                                                                                                        called Singular Values

                                                                                                        And Unitary (Rotation) Matrices

                                                                                                        (recall )

                                                                                                        ndX

                                                                                                        ndS

                                                                                                        )min(1 ndss

                                                                                                        ddU nnV

                                                                                                        IVVUU tt

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Singular Value Decomposition (SVD)

                                                                                                        For a Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        with Entries

                                                                                                        called Singular Values

                                                                                                        And Unitary (Rotation) Matrices

                                                                                                        (recall )

                                                                                                        So That

                                                                                                        ndX

                                                                                                        ndS

                                                                                                        )min(1 ndss

                                                                                                        ddU nnV

                                                                                                        IVVUU tt tUSVX

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                        vVSUvVSUvX tt

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                        bull First rotate

                                                                                                        X

                                                                                                        vVSUvVSUvX tt

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                        bull First rotate

                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                        X

                                                                                                        vVSUvVSUvX tt

                                                                                                        is

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                        bull First rotate

                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                        bull Third rotate again

                                                                                                        X

                                                                                                        vVSUvVSUvX tt

                                                                                                        is

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                        bull First rotate

                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                        bull Third rotate again

                                                                                                        bull ie have diagonalized the transformation

                                                                                                        X

                                                                                                        vVSUvVSUvX tt

                                                                                                        is

                                                                                                        SVD Compact Representation

                                                                                                        Useful Labeling

                                                                                                        Singular Values in Increasing Order

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        )min(1 dnss

                                                                                                        SVD Compact Representation

                                                                                                        Useful Labeling

                                                                                                        Singular Values in Increasing Order

                                                                                                        Note singular values = 0 can be omitted

                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        )min(1 dnss

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        r

                                                                                                        SVD Compact Representation

                                                                                                        Useful Labeling

                                                                                                        Singular Values in Increasing Order

                                                                                                        Note singular values = 0 can be omitted

                                                                                                        Let = of positive singular values

                                                                                                        )min(1 dnss

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        r

                                                                                                        SVD Compact Representation

                                                                                                        Useful Labeling

                                                                                                        Singular Values in Increasing Order

                                                                                                        Note singular values = 0 can be omitted

                                                                                                        Let = of positive singular values

                                                                                                        Then

                                                                                                        Where are truncations of

                                                                                                        trnrrrd VSUX

                                                                                                        VSU

                                                                                                        )min(1 dnss

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Full Representation

                                                                                                        =

                                                                                                        Graphics Display Assumes

                                                                                                        ndX ddU ndS nn

                                                                                                        tV

                                                                                                        nd

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Full Representation

                                                                                                        =

                                                                                                        Full Rank Basis Matrix

                                                                                                        ndX ddU ndS nn

                                                                                                        tV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Full Representation

                                                                                                        =

                                                                                                        Full Rank Basis Matrix

                                                                                                        All 0s in Bottom

                                                                                                        ndX ddU ndS nn

                                                                                                        tV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Reduced Representation

                                                                                                        =

                                                                                                        These Columns Get 0ed Out

                                                                                                        ndX ddU nnS nn

                                                                                                        tV

                                                                                                        nnd 0

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Reduced Representation

                                                                                                        =ndX ndU nnS nn

                                                                                                        tV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Reduced Representation

                                                                                                        =

                                                                                                        Also Some of These May be 0

                                                                                                        ndX ndU nnS nn

                                                                                                        tV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Compact Representation

                                                                                                        = ndX rdU

                                                                                                        rrS nrtV

                                                                                                        0

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Compact Representation

                                                                                                        =

                                                                                                        These Get 0ed Out

                                                                                                        ndX rdU

                                                                                                        rrS nrtV

                                                                                                        0

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        SVD Compact Representation

                                                                                                        = ndX rdU

                                                                                                        rrS nrtV

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition

                                                                                                        For a (Symmetric) Square Matrix

                                                                                                        Find

                                                                                                        ddX

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition

                                                                                                        For a (Symmetric) Square Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        Called Eigenvalues

                                                                                                        Convenient Ordering

                                                                                                        ddX

                                                                                                        d

                                                                                                        D

                                                                                                        0

                                                                                                        01

                                                                                                        n 1

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition

                                                                                                        For a (Symmetric) Square Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        And an Orthonormal Matrix

                                                                                                        (ie )

                                                                                                        ddX

                                                                                                        d

                                                                                                        D

                                                                                                        0

                                                                                                        01

                                                                                                        ddB

                                                                                                        ddtt IBBBB

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition

                                                                                                        For a (Symmetric) Square Matrix

                                                                                                        Find a Diagonal Matrix

                                                                                                        And an Orthonormal Matrix

                                                                                                        (ie )

                                                                                                        So that ie

                                                                                                        ddX

                                                                                                        d

                                                                                                        D

                                                                                                        0

                                                                                                        01

                                                                                                        ddB

                                                                                                        ddtt IBBBB

                                                                                                        DBBX tBDBX

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                        (looks similar)

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                        Complex (uses )

                                                                                                        VU

                                                                                                        1i

                                                                                                        Review of Linear Algebra (Cont)

                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                        VU

                                                                                                        X

                                                                                                        BVU

                                                                                                        1i

                                                                                                        • Participant Presentations
                                                                                                        • PCA to find clusters
                                                                                                        • Statistical Smoothing
                                                                                                        • Kernel Density Estimation
                                                                                                        • Scatterplot Smoothing
                                                                                                        • Statistical Smoothing (2)
                                                                                                        • SiZer Background
                                                                                                        • SiZer Background (2)
                                                                                                        • SiZer Background (3)
                                                                                                        • SiZer Background (4)
                                                                                                        • SiZer Background (5)
                                                                                                        • SiZer Overview
                                                                                                        • PCA to find clusters (2)
                                                                                                        • PCA to find clusters (3)
                                                                                                        • PCA to find clusters (4)
                                                                                                        • PCA to find clusters (5)
                                                                                                        • PCA to find clusters (6)
                                                                                                        • PCA to find clusters (7)
                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                        • Frequency 2 Analysis
                                                                                                        • Frequency 2 Analysis (2)
                                                                                                        • Frequency 2 Analysis (3)
                                                                                                        • Yeast Cell Cycle
                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                        • Reclassification of Major Genes
                                                                                                        • Compare to Previous Classifrsquon
                                                                                                        • New Subpopulation View
                                                                                                        • New Subpopulation View (2)
                                                                                                        • Detailed Look at PCA
                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                        • An Interesting Historical Note
                                                                                                        • An Interesting Historical Note (2)
                                                                                                        • Detailed Look at PCA (2)
                                                                                                        • Detailed Look at PCA (3)
                                                                                                        • Review of Linear Algebra
                                                                                                        • Review of Linear Algebra (2)
                                                                                                        • Review of Linear Algebra (3)
                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                                          where

                                                                                                          n

                                                                                                          iii aBva

                                                                                                          1

                                                                                                          na

                                                                                                          a

                                                                                                          a 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                                          where

                                                                                                          Note Right Multiplication Gives

                                                                                                          Linear Combination of Column Vectors

                                                                                                          n

                                                                                                          iii aBva

                                                                                                          1

                                                                                                          na

                                                                                                          a

                                                                                                          a 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Then linear combo is a matrix multiplicatrsquon

                                                                                                          where

                                                                                                          Check sizes

                                                                                                          n

                                                                                                          iii aBva

                                                                                                          1

                                                                                                          na

                                                                                                          a

                                                                                                          a 1

                                                                                                          )1()(1 nndd

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                          For matrices

                                                                                                          Define the Matrix Product

                                                                                                          mkk

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          nmm

                                                                                                          n

                                                                                                          bb

                                                                                                          bb

                                                                                                          B

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iniik

                                                                                                          m

                                                                                                          iiik

                                                                                                          m

                                                                                                          inii

                                                                                                          m

                                                                                                          iii

                                                                                                          baba

                                                                                                          baba

                                                                                                          AB

                                                                                                          1

                                                                                                          11

                                                                                                          11

                                                                                                          111

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                          For matrices

                                                                                                          Define the Matrix Product

                                                                                                          (Inner Products of Rows With Columns )

                                                                                                          mkk

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          nmm

                                                                                                          n

                                                                                                          bb

                                                                                                          bb

                                                                                                          B

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iniik

                                                                                                          m

                                                                                                          iiik

                                                                                                          m

                                                                                                          inii

                                                                                                          m

                                                                                                          iii

                                                                                                          baba

                                                                                                          baba

                                                                                                          AB

                                                                                                          1

                                                                                                          11

                                                                                                          11

                                                                                                          111

                                                                                                          A B

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                          For matrices

                                                                                                          Define the Matrix Product

                                                                                                          (Inner Products of Rows With Columns )

                                                                                                          (Composition of Linear Transformations)

                                                                                                          mkk

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          nmm

                                                                                                          n

                                                                                                          bb

                                                                                                          bb

                                                                                                          B

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iniik

                                                                                                          m

                                                                                                          iiik

                                                                                                          m

                                                                                                          inii

                                                                                                          m

                                                                                                          iii

                                                                                                          baba

                                                                                                          baba

                                                                                                          AB

                                                                                                          1

                                                                                                          11

                                                                                                          11

                                                                                                          111

                                                                                                          A B

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                          For matrices

                                                                                                          Define the Matrix Product

                                                                                                          (Inner Products of Rows With Columns )

                                                                                                          (Composition of Linear Transformations)

                                                                                                          Often Useful to Check Sizes

                                                                                                          mkk

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          nmm

                                                                                                          n

                                                                                                          bb

                                                                                                          bb

                                                                                                          B

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iniik

                                                                                                          m

                                                                                                          iiik

                                                                                                          m

                                                                                                          inii

                                                                                                          m

                                                                                                          iii

                                                                                                          baba

                                                                                                          baba

                                                                                                          AB

                                                                                                          1

                                                                                                          11

                                                                                                          11

                                                                                                          111

                                                                                                          nmmknk

                                                                                                          A B

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                          For matrices

                                                                                                          Define the Matrix Product

                                                                                                          (Inner Products of Rows With Columns )

                                                                                                          (Composition of Linear Transformations)

                                                                                                          Often Useful to Check Sizes

                                                                                                          mkk

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          nmm

                                                                                                          n

                                                                                                          bb

                                                                                                          bb

                                                                                                          B

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iniik

                                                                                                          m

                                                                                                          iiik

                                                                                                          m

                                                                                                          inii

                                                                                                          m

                                                                                                          iii

                                                                                                          baba

                                                                                                          baba

                                                                                                          AB

                                                                                                          1

                                                                                                          11

                                                                                                          11

                                                                                                          111

                                                                                                          nmmknk

                                                                                                          A B

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Matrix Trace

                                                                                                          bull For a Square Matrix

                                                                                                          bull Define

                                                                                                          mmm

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iiiaAtr

                                                                                                          1)(

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Matrix Trace

                                                                                                          bull For a Square Matrix

                                                                                                          bull Define

                                                                                                          bull Trace Commutes with Matrix Multiplication

                                                                                                          mmm

                                                                                                          m

                                                                                                          aa

                                                                                                          aa

                                                                                                          A

                                                                                                          1

                                                                                                          111

                                                                                                          m

                                                                                                          iiiaAtr

                                                                                                          1)(

                                                                                                          BAtrABtr

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                          bull (Use Basis Above)

                                                                                                          bull eg dim of a line is 1

                                                                                                          bull eg dim of a plane is 2

                                                                                                          dd dim

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                          bull (Use Basis Above)

                                                                                                          bull eg dim of a line is 1

                                                                                                          bull eg dim of a plane is 2

                                                                                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                          (in Statistical Uses eg ANOVA)

                                                                                                          dd dim

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Norm of a Vector

                                                                                                          bull in d 21

                                                                                                          21

                                                                                                          1

                                                                                                          2 xxxx td

                                                                                                          jj

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Norm of a Vector

                                                                                                          bull in

                                                                                                          bull Idea length of the vector

                                                                                                          d 2121

                                                                                                          1

                                                                                                          2 xxxx td

                                                                                                          jj

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Norm of a Vector

                                                                                                          bull in

                                                                                                          bull Idea length of the vector

                                                                                                          bull Note strange properties for high

                                                                                                          eg ldquolength of diagonal of unit cuberdquo =

                                                                                                          d 2121

                                                                                                          1

                                                                                                          2 xxxx td

                                                                                                          jj

                                                                                                          d

                                                                                                          d

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Norm of a Vector (cont)

                                                                                                          bull Length Normalized Vector

                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                          amp is a Direction Vector)

                                                                                                          x

                                                                                                          x

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Norm of a Vector (cont)

                                                                                                          bull Length Normalized Vector

                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                          amp is a Direction Vector)

                                                                                                          bull Define Distance as

                                                                                                          x

                                                                                                          x

                                                                                                          yxyxyxyxd t

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Inner (Dot Scalar) Product

                                                                                                          bull for Vectors and

                                                                                                          yxyxyx td

                                                                                                          jjj

                                                                                                          1

                                                                                                          x y

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Inner (Dot Scalar) Product

                                                                                                          bull for Vectors and

                                                                                                          bull Related to Norm via

                                                                                                          yxyxyx td

                                                                                                          jjj

                                                                                                          1

                                                                                                          x y

                                                                                                          21 xxx

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                          yyxx

                                                                                                          yx

                                                                                                          yx

                                                                                                          yxyxangle

                                                                                                          tt

                                                                                                          t

                                                                                                          11 cos

                                                                                                          cos

                                                                                                          x y

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                          if and only if

                                                                                                          yyxx

                                                                                                          yx

                                                                                                          yx

                                                                                                          yxyxangle

                                                                                                          tt

                                                                                                          t

                                                                                                          11 cos

                                                                                                          cos

                                                                                                          x y

                                                                                                          yx 0 yx

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis

                                                                                                          bull All Orthogonal to each other

                                                                                                          ie for

                                                                                                          bull All have Length 1

                                                                                                          ie for

                                                                                                          nvv 1

                                                                                                          1 ii vv

                                                                                                          0 ii vv ii

                                                                                                          ni 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis (cont)

                                                                                                          bull Spectral Representation

                                                                                                          where

                                                                                                          nvv 1

                                                                                                          n

                                                                                                          iii vax

                                                                                                          1

                                                                                                          ii vxa

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis (cont)

                                                                                                          bull Spectral Representation

                                                                                                          where

                                                                                                          (Coefficient is Inner Product Cool Notation)

                                                                                                          nvv 1

                                                                                                          n

                                                                                                          iii vax

                                                                                                          1

                                                                                                          ii vxa

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis (cont)

                                                                                                          bull Spectral Representation

                                                                                                          where

                                                                                                          Check

                                                                                                          nvv 1

                                                                                                          n

                                                                                                          iii vax

                                                                                                          1

                                                                                                          ii vxa

                                                                                                          iii

                                                                                                          n

                                                                                                          iii

                                                                                                          n

                                                                                                          iiii avvavvavx

                                                                                                          1

                                                                                                          1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis (cont)

                                                                                                          bull Spectral Representation

                                                                                                          where

                                                                                                          Check

                                                                                                          bull Matrix Notation where ie

                                                                                                          For the Basis Matrix

                                                                                                          nvv 1

                                                                                                          n

                                                                                                          iii vax

                                                                                                          1

                                                                                                          ii vxa

                                                                                                          iii

                                                                                                          n

                                                                                                          iii

                                                                                                          n

                                                                                                          iiii avvavvavx

                                                                                                          1

                                                                                                          1

                                                                                                          aBx Bxa tt xBa t

                                                                                                          nvvB 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Orthonormal Basis (cont)

                                                                                                          bull Spectral Representation

                                                                                                          where

                                                                                                          Check

                                                                                                          bull Matrix Notation where ie

                                                                                                          is called transform of

                                                                                                          (eg Fourier or Wavelet)

                                                                                                          nvv 1

                                                                                                          n

                                                                                                          iii vax

                                                                                                          1

                                                                                                          ii vxa

                                                                                                          iii

                                                                                                          n

                                                                                                          iii

                                                                                                          n

                                                                                                          iiii avvavvavx

                                                                                                          1

                                                                                                          1

                                                                                                          aBx Bxa tt xBa t

                                                                                                          xa

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Parseval identity for

                                                                                                          in subsp genrsquod by o n basis

                                                                                                          x

                                                                                                          nvv 1

                                                                                                          2

                                                                                                          1

                                                                                                          22

                                                                                                          1

                                                                                                          2 aavxx

                                                                                                          n

                                                                                                          ii

                                                                                                          n

                                                                                                          ii

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Parseval identity for

                                                                                                          in subsp genrsquod by o n basis

                                                                                                          bull Pythagorean theorem

                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                          bull ANOVA - sums of squares

                                                                                                          x

                                                                                                          nvv 1

                                                                                                          2

                                                                                                          1

                                                                                                          22

                                                                                                          1

                                                                                                          2 aavxx

                                                                                                          n

                                                                                                          ii

                                                                                                          n

                                                                                                          ii

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Parseval identity for

                                                                                                          in subsp genrsquod by o n basis

                                                                                                          bull Pythagorean theorem

                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                          bull ANOVA - sums of squares

                                                                                                          bull Transform has same length as

                                                                                                          ie ldquorotation in rdquo

                                                                                                          x

                                                                                                          nvv 1

                                                                                                          2

                                                                                                          1

                                                                                                          22

                                                                                                          1

                                                                                                          2 aavxx

                                                                                                          n

                                                                                                          ii

                                                                                                          n

                                                                                                          ii

                                                                                                          a xd

                                                                                                          Projection of a Vector onto a Subspace

                                                                                                          bull Idea Member of that is Closest to

                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                          xV

                                                                                                          V

                                                                                                          Projection of a Vector onto a Subspace

                                                                                                          bull Idea Member of that is Closest to

                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                          bull Find that Solves

                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                          xV

                                                                                                          V

                                                                                                          VxPV vxVv

                                                                                                          min

                                                                                                          Projection of a Vector onto a Subspace

                                                                                                          bull Idea Member of that is Closest to

                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                          bull Find that Solves

                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                          bull For Inner Product (Hilbert) Space

                                                                                                          Exists and is Unique

                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                          xV

                                                                                                          V

                                                                                                          VxPV vxVv

                                                                                                          min

                                                                                                          xPV

                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                          bull General Solution in for Basis Matrix

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          d VB

                                                                                                          xBBBBxP tVV

                                                                                                          tVVV

                                                                                                          1

                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                          bull General Solution in for Basis Matrix

                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                          (thus projection is another linear operation)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          d VB

                                                                                                          xBBBBxP tVV

                                                                                                          tVVV

                                                                                                          1

                                                                                                          tVV

                                                                                                          tVVV BBBBP

                                                                                                          1

                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                          bull General Solution in for Basis Matrix

                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                          (thus projection is another linear operation)

                                                                                                          (note same operation underlies least squares)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          d VB

                                                                                                          xBBBBxP tVV

                                                                                                          tVVV

                                                                                                          1

                                                                                                          tVV

                                                                                                          tVVV BBBBP

                                                                                                          1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Projection using Orthonormal Basis

                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                          nnVtV IBB

                                                                                                          10

                                                                                                          01

                                                                                                          1

                                                                                                          111

                                                                                                          1

                                                                                                          1

                                                                                                          nnn

                                                                                                          n

                                                                                                          ntn

                                                                                                          t

                                                                                                          vvvv

                                                                                                          vvvv

                                                                                                          vv

                                                                                                          v

                                                                                                          v

                                                                                                          nvv 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Projection using Orthonormal Basis

                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                          bull So =

                                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                          (Recall Right Multrsquon)

                                                                                                          nnVtV IBB

                                                                                                          10

                                                                                                          01

                                                                                                          1

                                                                                                          111

                                                                                                          1

                                                                                                          1

                                                                                                          nnn

                                                                                                          n

                                                                                                          ntn

                                                                                                          t

                                                                                                          vvvv

                                                                                                          vvvv

                                                                                                          vv

                                                                                                          v

                                                                                                          v

                                                                                                          xBBxP tVVV

                                                                                                          x V

                                                                                                          nvv 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                          bull For Orthogonal Complement

                                                                                                          and

                                                                                                          V

                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                          bull For Orthogonal Complement

                                                                                                          and

                                                                                                          bull Parseval Inequality

                                                                                                          V

                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                          2

                                                                                                          1

                                                                                                          22

                                                                                                          1

                                                                                                          22 aavxxxP

                                                                                                          n

                                                                                                          ii

                                                                                                          n

                                                                                                          iiV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          (Real) Unitary Matrices with

                                                                                                          bull Orthonormal Basis Matrix

                                                                                                          (So All of Above Applies)

                                                                                                          ddU IUU t

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          (Real) Unitary Matrices with

                                                                                                          bull Orthonormal Basis Matrix

                                                                                                          (So All of Above Applies)

                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                          ddU IUU t

                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                          i ii 2

                                                                                                          1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          (Real) Unitary Matrices with

                                                                                                          bull Orthonormal Basis Matrix

                                                                                                          (So All of Above Applies)

                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                          ddU IUU t

                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                          i ii 2

                                                                                                          1

                                                                                                          U

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Singular Value Decomposition (SVD)

                                                                                                          For a Matrix

                                                                                                          Find

                                                                                                          ndX

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Singular Value Decomposition (SVD)

                                                                                                          For a Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          with Entries

                                                                                                          called Singular Values

                                                                                                          ndX

                                                                                                          ndS

                                                                                                          )min(1 ndss

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Singular Value Decomposition (SVD)

                                                                                                          For a Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          with Entries

                                                                                                          called Singular Values

                                                                                                          And Unitary (Rotation) Matrices

                                                                                                          (recall )

                                                                                                          ndX

                                                                                                          ndS

                                                                                                          )min(1 ndss

                                                                                                          ddU nnV

                                                                                                          IVVUU tt

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Singular Value Decomposition (SVD)

                                                                                                          For a Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          with Entries

                                                                                                          called Singular Values

                                                                                                          And Unitary (Rotation) Matrices

                                                                                                          (recall )

                                                                                                          So That

                                                                                                          ndX

                                                                                                          ndS

                                                                                                          )min(1 ndss

                                                                                                          ddU nnV

                                                                                                          IVVUU tt tUSVX

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                          vVSUvVSUvX tt

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                          bull First rotate

                                                                                                          X

                                                                                                          vVSUvVSUvX tt

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                          bull First rotate

                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                          X

                                                                                                          vVSUvVSUvX tt

                                                                                                          is

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                          bull First rotate

                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                          bull Third rotate again

                                                                                                          X

                                                                                                          vVSUvVSUvX tt

                                                                                                          is

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                          bull First rotate

                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                          bull Third rotate again

                                                                                                          bull ie have diagonalized the transformation

                                                                                                          X

                                                                                                          vVSUvVSUvX tt

                                                                                                          is

                                                                                                          SVD Compact Representation

                                                                                                          Useful Labeling

                                                                                                          Singular Values in Increasing Order

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          )min(1 dnss

                                                                                                          SVD Compact Representation

                                                                                                          Useful Labeling

                                                                                                          Singular Values in Increasing Order

                                                                                                          Note singular values = 0 can be omitted

                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          )min(1 dnss

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          r

                                                                                                          SVD Compact Representation

                                                                                                          Useful Labeling

                                                                                                          Singular Values in Increasing Order

                                                                                                          Note singular values = 0 can be omitted

                                                                                                          Let = of positive singular values

                                                                                                          )min(1 dnss

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          r

                                                                                                          SVD Compact Representation

                                                                                                          Useful Labeling

                                                                                                          Singular Values in Increasing Order

                                                                                                          Note singular values = 0 can be omitted

                                                                                                          Let = of positive singular values

                                                                                                          Then

                                                                                                          Where are truncations of

                                                                                                          trnrrrd VSUX

                                                                                                          VSU

                                                                                                          )min(1 dnss

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Full Representation

                                                                                                          =

                                                                                                          Graphics Display Assumes

                                                                                                          ndX ddU ndS nn

                                                                                                          tV

                                                                                                          nd

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Full Representation

                                                                                                          =

                                                                                                          Full Rank Basis Matrix

                                                                                                          ndX ddU ndS nn

                                                                                                          tV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Full Representation

                                                                                                          =

                                                                                                          Full Rank Basis Matrix

                                                                                                          All 0s in Bottom

                                                                                                          ndX ddU ndS nn

                                                                                                          tV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Reduced Representation

                                                                                                          =

                                                                                                          These Columns Get 0ed Out

                                                                                                          ndX ddU nnS nn

                                                                                                          tV

                                                                                                          nnd 0

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Reduced Representation

                                                                                                          =ndX ndU nnS nn

                                                                                                          tV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Reduced Representation

                                                                                                          =

                                                                                                          Also Some of These May be 0

                                                                                                          ndX ndU nnS nn

                                                                                                          tV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Compact Representation

                                                                                                          = ndX rdU

                                                                                                          rrS nrtV

                                                                                                          0

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Compact Representation

                                                                                                          =

                                                                                                          These Get 0ed Out

                                                                                                          ndX rdU

                                                                                                          rrS nrtV

                                                                                                          0

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          SVD Compact Representation

                                                                                                          = ndX rdU

                                                                                                          rrS nrtV

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition

                                                                                                          For a (Symmetric) Square Matrix

                                                                                                          Find

                                                                                                          ddX

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition

                                                                                                          For a (Symmetric) Square Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          Called Eigenvalues

                                                                                                          Convenient Ordering

                                                                                                          ddX

                                                                                                          d

                                                                                                          D

                                                                                                          0

                                                                                                          01

                                                                                                          n 1

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition

                                                                                                          For a (Symmetric) Square Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          And an Orthonormal Matrix

                                                                                                          (ie )

                                                                                                          ddX

                                                                                                          d

                                                                                                          D

                                                                                                          0

                                                                                                          01

                                                                                                          ddB

                                                                                                          ddtt IBBBB

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition

                                                                                                          For a (Symmetric) Square Matrix

                                                                                                          Find a Diagonal Matrix

                                                                                                          And an Orthonormal Matrix

                                                                                                          (ie )

                                                                                                          So that ie

                                                                                                          ddX

                                                                                                          d

                                                                                                          D

                                                                                                          0

                                                                                                          01

                                                                                                          ddB

                                                                                                          ddtt IBBBB

                                                                                                          DBBX tBDBX

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                          (looks similar)

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                          Complex (uses )

                                                                                                          VU

                                                                                                          1i

                                                                                                          Review of Linear Algebra (Cont)

                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                          VU

                                                                                                          X

                                                                                                          BVU

                                                                                                          1i

                                                                                                          • Participant Presentations
                                                                                                          • PCA to find clusters
                                                                                                          • Statistical Smoothing
                                                                                                          • Kernel Density Estimation
                                                                                                          • Scatterplot Smoothing
                                                                                                          • Statistical Smoothing (2)
                                                                                                          • SiZer Background
                                                                                                          • SiZer Background (2)
                                                                                                          • SiZer Background (3)
                                                                                                          • SiZer Background (4)
                                                                                                          • SiZer Background (5)
                                                                                                          • SiZer Overview
                                                                                                          • PCA to find clusters (2)
                                                                                                          • PCA to find clusters (3)
                                                                                                          • PCA to find clusters (4)
                                                                                                          • PCA to find clusters (5)
                                                                                                          • PCA to find clusters (6)
                                                                                                          • PCA to find clusters (7)
                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                          • Frequency 2 Analysis
                                                                                                          • Frequency 2 Analysis (2)
                                                                                                          • Frequency 2 Analysis (3)
                                                                                                          • Yeast Cell Cycle
                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                          • Reclassification of Major Genes
                                                                                                          • Compare to Previous Classifrsquon
                                                                                                          • New Subpopulation View
                                                                                                          • New Subpopulation View (2)
                                                                                                          • Detailed Look at PCA
                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                          • An Interesting Historical Note
                                                                                                          • An Interesting Historical Note (2)
                                                                                                          • Detailed Look at PCA (2)
                                                                                                          • Detailed Look at PCA (3)
                                                                                                          • Review of Linear Algebra
                                                                                                          • Review of Linear Algebra (2)
                                                                                                          • Review of Linear Algebra (3)
                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                                                            where

                                                                                                            Note Right Multiplication Gives

                                                                                                            Linear Combination of Column Vectors

                                                                                                            n

                                                                                                            iii aBva

                                                                                                            1

                                                                                                            na

                                                                                                            a

                                                                                                            a 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Then linear combo is a matrix multiplicatrsquon

                                                                                                            where

                                                                                                            Check sizes

                                                                                                            n

                                                                                                            iii aBva

                                                                                                            1

                                                                                                            na

                                                                                                            a

                                                                                                            a 1

                                                                                                            )1()(1 nndd

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                            For matrices

                                                                                                            Define the Matrix Product

                                                                                                            mkk

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            nmm

                                                                                                            n

                                                                                                            bb

                                                                                                            bb

                                                                                                            B

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iniik

                                                                                                            m

                                                                                                            iiik

                                                                                                            m

                                                                                                            inii

                                                                                                            m

                                                                                                            iii

                                                                                                            baba

                                                                                                            baba

                                                                                                            AB

                                                                                                            1

                                                                                                            11

                                                                                                            11

                                                                                                            111

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                            For matrices

                                                                                                            Define the Matrix Product

                                                                                                            (Inner Products of Rows With Columns )

                                                                                                            mkk

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            nmm

                                                                                                            n

                                                                                                            bb

                                                                                                            bb

                                                                                                            B

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iniik

                                                                                                            m

                                                                                                            iiik

                                                                                                            m

                                                                                                            inii

                                                                                                            m

                                                                                                            iii

                                                                                                            baba

                                                                                                            baba

                                                                                                            AB

                                                                                                            1

                                                                                                            11

                                                                                                            11

                                                                                                            111

                                                                                                            A B

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                            For matrices

                                                                                                            Define the Matrix Product

                                                                                                            (Inner Products of Rows With Columns )

                                                                                                            (Composition of Linear Transformations)

                                                                                                            mkk

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            nmm

                                                                                                            n

                                                                                                            bb

                                                                                                            bb

                                                                                                            B

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iniik

                                                                                                            m

                                                                                                            iiik

                                                                                                            m

                                                                                                            inii

                                                                                                            m

                                                                                                            iii

                                                                                                            baba

                                                                                                            baba

                                                                                                            AB

                                                                                                            1

                                                                                                            11

                                                                                                            11

                                                                                                            111

                                                                                                            A B

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                            For matrices

                                                                                                            Define the Matrix Product

                                                                                                            (Inner Products of Rows With Columns )

                                                                                                            (Composition of Linear Transformations)

                                                                                                            Often Useful to Check Sizes

                                                                                                            mkk

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            nmm

                                                                                                            n

                                                                                                            bb

                                                                                                            bb

                                                                                                            B

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iniik

                                                                                                            m

                                                                                                            iiik

                                                                                                            m

                                                                                                            inii

                                                                                                            m

                                                                                                            iii

                                                                                                            baba

                                                                                                            baba

                                                                                                            AB

                                                                                                            1

                                                                                                            11

                                                                                                            11

                                                                                                            111

                                                                                                            nmmknk

                                                                                                            A B

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                            For matrices

                                                                                                            Define the Matrix Product

                                                                                                            (Inner Products of Rows With Columns )

                                                                                                            (Composition of Linear Transformations)

                                                                                                            Often Useful to Check Sizes

                                                                                                            mkk

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            nmm

                                                                                                            n

                                                                                                            bb

                                                                                                            bb

                                                                                                            B

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iniik

                                                                                                            m

                                                                                                            iiik

                                                                                                            m

                                                                                                            inii

                                                                                                            m

                                                                                                            iii

                                                                                                            baba

                                                                                                            baba

                                                                                                            AB

                                                                                                            1

                                                                                                            11

                                                                                                            11

                                                                                                            111

                                                                                                            nmmknk

                                                                                                            A B

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Matrix Trace

                                                                                                            bull For a Square Matrix

                                                                                                            bull Define

                                                                                                            mmm

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iiiaAtr

                                                                                                            1)(

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Matrix Trace

                                                                                                            bull For a Square Matrix

                                                                                                            bull Define

                                                                                                            bull Trace Commutes with Matrix Multiplication

                                                                                                            mmm

                                                                                                            m

                                                                                                            aa

                                                                                                            aa

                                                                                                            A

                                                                                                            1

                                                                                                            111

                                                                                                            m

                                                                                                            iiiaAtr

                                                                                                            1)(

                                                                                                            BAtrABtr

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                            bull (Use Basis Above)

                                                                                                            bull eg dim of a line is 1

                                                                                                            bull eg dim of a plane is 2

                                                                                                            dd dim

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                            bull (Use Basis Above)

                                                                                                            bull eg dim of a line is 1

                                                                                                            bull eg dim of a plane is 2

                                                                                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                            (in Statistical Uses eg ANOVA)

                                                                                                            dd dim

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Norm of a Vector

                                                                                                            bull in d 21

                                                                                                            21

                                                                                                            1

                                                                                                            2 xxxx td

                                                                                                            jj

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Norm of a Vector

                                                                                                            bull in

                                                                                                            bull Idea length of the vector

                                                                                                            d 2121

                                                                                                            1

                                                                                                            2 xxxx td

                                                                                                            jj

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Norm of a Vector

                                                                                                            bull in

                                                                                                            bull Idea length of the vector

                                                                                                            bull Note strange properties for high

                                                                                                            eg ldquolength of diagonal of unit cuberdquo =

                                                                                                            d 2121

                                                                                                            1

                                                                                                            2 xxxx td

                                                                                                            jj

                                                                                                            d

                                                                                                            d

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Norm of a Vector (cont)

                                                                                                            bull Length Normalized Vector

                                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                                            amp is a Direction Vector)

                                                                                                            x

                                                                                                            x

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Norm of a Vector (cont)

                                                                                                            bull Length Normalized Vector

                                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                                            amp is a Direction Vector)

                                                                                                            bull Define Distance as

                                                                                                            x

                                                                                                            x

                                                                                                            yxyxyxyxd t

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Inner (Dot Scalar) Product

                                                                                                            bull for Vectors and

                                                                                                            yxyxyx td

                                                                                                            jjj

                                                                                                            1

                                                                                                            x y

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Inner (Dot Scalar) Product

                                                                                                            bull for Vectors and

                                                                                                            bull Related to Norm via

                                                                                                            yxyxyx td

                                                                                                            jjj

                                                                                                            1

                                                                                                            x y

                                                                                                            21 xxx

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                            yyxx

                                                                                                            yx

                                                                                                            yx

                                                                                                            yxyxangle

                                                                                                            tt

                                                                                                            t

                                                                                                            11 cos

                                                                                                            cos

                                                                                                            x y

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                            if and only if

                                                                                                            yyxx

                                                                                                            yx

                                                                                                            yx

                                                                                                            yxyxangle

                                                                                                            tt

                                                                                                            t

                                                                                                            11 cos

                                                                                                            cos

                                                                                                            x y

                                                                                                            yx 0 yx

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis

                                                                                                            bull All Orthogonal to each other

                                                                                                            ie for

                                                                                                            bull All have Length 1

                                                                                                            ie for

                                                                                                            nvv 1

                                                                                                            1 ii vv

                                                                                                            0 ii vv ii

                                                                                                            ni 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis (cont)

                                                                                                            bull Spectral Representation

                                                                                                            where

                                                                                                            nvv 1

                                                                                                            n

                                                                                                            iii vax

                                                                                                            1

                                                                                                            ii vxa

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis (cont)

                                                                                                            bull Spectral Representation

                                                                                                            where

                                                                                                            (Coefficient is Inner Product Cool Notation)

                                                                                                            nvv 1

                                                                                                            n

                                                                                                            iii vax

                                                                                                            1

                                                                                                            ii vxa

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis (cont)

                                                                                                            bull Spectral Representation

                                                                                                            where

                                                                                                            Check

                                                                                                            nvv 1

                                                                                                            n

                                                                                                            iii vax

                                                                                                            1

                                                                                                            ii vxa

                                                                                                            iii

                                                                                                            n

                                                                                                            iii

                                                                                                            n

                                                                                                            iiii avvavvavx

                                                                                                            1

                                                                                                            1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis (cont)

                                                                                                            bull Spectral Representation

                                                                                                            where

                                                                                                            Check

                                                                                                            bull Matrix Notation where ie

                                                                                                            For the Basis Matrix

                                                                                                            nvv 1

                                                                                                            n

                                                                                                            iii vax

                                                                                                            1

                                                                                                            ii vxa

                                                                                                            iii

                                                                                                            n

                                                                                                            iii

                                                                                                            n

                                                                                                            iiii avvavvavx

                                                                                                            1

                                                                                                            1

                                                                                                            aBx Bxa tt xBa t

                                                                                                            nvvB 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Orthonormal Basis (cont)

                                                                                                            bull Spectral Representation

                                                                                                            where

                                                                                                            Check

                                                                                                            bull Matrix Notation where ie

                                                                                                            is called transform of

                                                                                                            (eg Fourier or Wavelet)

                                                                                                            nvv 1

                                                                                                            n

                                                                                                            iii vax

                                                                                                            1

                                                                                                            ii vxa

                                                                                                            iii

                                                                                                            n

                                                                                                            iii

                                                                                                            n

                                                                                                            iiii avvavvavx

                                                                                                            1

                                                                                                            1

                                                                                                            aBx Bxa tt xBa t

                                                                                                            xa

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Parseval identity for

                                                                                                            in subsp genrsquod by o n basis

                                                                                                            x

                                                                                                            nvv 1

                                                                                                            2

                                                                                                            1

                                                                                                            22

                                                                                                            1

                                                                                                            2 aavxx

                                                                                                            n

                                                                                                            ii

                                                                                                            n

                                                                                                            ii

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Parseval identity for

                                                                                                            in subsp genrsquod by o n basis

                                                                                                            bull Pythagorean theorem

                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                            bull ANOVA - sums of squares

                                                                                                            x

                                                                                                            nvv 1

                                                                                                            2

                                                                                                            1

                                                                                                            22

                                                                                                            1

                                                                                                            2 aavxx

                                                                                                            n

                                                                                                            ii

                                                                                                            n

                                                                                                            ii

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Parseval identity for

                                                                                                            in subsp genrsquod by o n basis

                                                                                                            bull Pythagorean theorem

                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                            bull ANOVA - sums of squares

                                                                                                            bull Transform has same length as

                                                                                                            ie ldquorotation in rdquo

                                                                                                            x

                                                                                                            nvv 1

                                                                                                            2

                                                                                                            1

                                                                                                            22

                                                                                                            1

                                                                                                            2 aavxx

                                                                                                            n

                                                                                                            ii

                                                                                                            n

                                                                                                            ii

                                                                                                            a xd

                                                                                                            Projection of a Vector onto a Subspace

                                                                                                            bull Idea Member of that is Closest to

                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                            xV

                                                                                                            V

                                                                                                            Projection of a Vector onto a Subspace

                                                                                                            bull Idea Member of that is Closest to

                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                            bull Find that Solves

                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                            xV

                                                                                                            V

                                                                                                            VxPV vxVv

                                                                                                            min

                                                                                                            Projection of a Vector onto a Subspace

                                                                                                            bull Idea Member of that is Closest to

                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                            bull Find that Solves

                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                            bull For Inner Product (Hilbert) Space

                                                                                                            Exists and is Unique

                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                            xV

                                                                                                            V

                                                                                                            VxPV vxVv

                                                                                                            min

                                                                                                            xPV

                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                            bull General Solution in for Basis Matrix

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            d VB

                                                                                                            xBBBBxP tVV

                                                                                                            tVVV

                                                                                                            1

                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                            bull General Solution in for Basis Matrix

                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                            (thus projection is another linear operation)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            d VB

                                                                                                            xBBBBxP tVV

                                                                                                            tVVV

                                                                                                            1

                                                                                                            tVV

                                                                                                            tVVV BBBBP

                                                                                                            1

                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                            bull General Solution in for Basis Matrix

                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                            (thus projection is another linear operation)

                                                                                                            (note same operation underlies least squares)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            d VB

                                                                                                            xBBBBxP tVV

                                                                                                            tVVV

                                                                                                            1

                                                                                                            tVV

                                                                                                            tVVV BBBBP

                                                                                                            1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Projection using Orthonormal Basis

                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                            nnVtV IBB

                                                                                                            10

                                                                                                            01

                                                                                                            1

                                                                                                            111

                                                                                                            1

                                                                                                            1

                                                                                                            nnn

                                                                                                            n

                                                                                                            ntn

                                                                                                            t

                                                                                                            vvvv

                                                                                                            vvvv

                                                                                                            vv

                                                                                                            v

                                                                                                            v

                                                                                                            nvv 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Projection using Orthonormal Basis

                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                            bull So =

                                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                            (Recall Right Multrsquon)

                                                                                                            nnVtV IBB

                                                                                                            10

                                                                                                            01

                                                                                                            1

                                                                                                            111

                                                                                                            1

                                                                                                            1

                                                                                                            nnn

                                                                                                            n

                                                                                                            ntn

                                                                                                            t

                                                                                                            vvvv

                                                                                                            vvvv

                                                                                                            vv

                                                                                                            v

                                                                                                            v

                                                                                                            xBBxP tVVV

                                                                                                            x V

                                                                                                            nvv 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                            bull For Orthogonal Complement

                                                                                                            and

                                                                                                            V

                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                            bull For Orthogonal Complement

                                                                                                            and

                                                                                                            bull Parseval Inequality

                                                                                                            V

                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                            2

                                                                                                            1

                                                                                                            22

                                                                                                            1

                                                                                                            22 aavxxxP

                                                                                                            n

                                                                                                            ii

                                                                                                            n

                                                                                                            iiV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            (Real) Unitary Matrices with

                                                                                                            bull Orthonormal Basis Matrix

                                                                                                            (So All of Above Applies)

                                                                                                            ddU IUU t

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            (Real) Unitary Matrices with

                                                                                                            bull Orthonormal Basis Matrix

                                                                                                            (So All of Above Applies)

                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                            ddU IUU t

                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                            i ii 2

                                                                                                            1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            (Real) Unitary Matrices with

                                                                                                            bull Orthonormal Basis Matrix

                                                                                                            (So All of Above Applies)

                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                            ddU IUU t

                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                            i ii 2

                                                                                                            1

                                                                                                            U

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Singular Value Decomposition (SVD)

                                                                                                            For a Matrix

                                                                                                            Find

                                                                                                            ndX

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Singular Value Decomposition (SVD)

                                                                                                            For a Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            with Entries

                                                                                                            called Singular Values

                                                                                                            ndX

                                                                                                            ndS

                                                                                                            )min(1 ndss

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Singular Value Decomposition (SVD)

                                                                                                            For a Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            with Entries

                                                                                                            called Singular Values

                                                                                                            And Unitary (Rotation) Matrices

                                                                                                            (recall )

                                                                                                            ndX

                                                                                                            ndS

                                                                                                            )min(1 ndss

                                                                                                            ddU nnV

                                                                                                            IVVUU tt

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Singular Value Decomposition (SVD)

                                                                                                            For a Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            with Entries

                                                                                                            called Singular Values

                                                                                                            And Unitary (Rotation) Matrices

                                                                                                            (recall )

                                                                                                            So That

                                                                                                            ndX

                                                                                                            ndS

                                                                                                            )min(1 ndss

                                                                                                            ddU nnV

                                                                                                            IVVUU tt tUSVX

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                            vVSUvVSUvX tt

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                            bull First rotate

                                                                                                            X

                                                                                                            vVSUvVSUvX tt

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                            bull First rotate

                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                            X

                                                                                                            vVSUvVSUvX tt

                                                                                                            is

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                            bull First rotate

                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                            bull Third rotate again

                                                                                                            X

                                                                                                            vVSUvVSUvX tt

                                                                                                            is

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                            bull First rotate

                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                            bull Third rotate again

                                                                                                            bull ie have diagonalized the transformation

                                                                                                            X

                                                                                                            vVSUvVSUvX tt

                                                                                                            is

                                                                                                            SVD Compact Representation

                                                                                                            Useful Labeling

                                                                                                            Singular Values in Increasing Order

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            )min(1 dnss

                                                                                                            SVD Compact Representation

                                                                                                            Useful Labeling

                                                                                                            Singular Values in Increasing Order

                                                                                                            Note singular values = 0 can be omitted

                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            )min(1 dnss

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            r

                                                                                                            SVD Compact Representation

                                                                                                            Useful Labeling

                                                                                                            Singular Values in Increasing Order

                                                                                                            Note singular values = 0 can be omitted

                                                                                                            Let = of positive singular values

                                                                                                            )min(1 dnss

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            r

                                                                                                            SVD Compact Representation

                                                                                                            Useful Labeling

                                                                                                            Singular Values in Increasing Order

                                                                                                            Note singular values = 0 can be omitted

                                                                                                            Let = of positive singular values

                                                                                                            Then

                                                                                                            Where are truncations of

                                                                                                            trnrrrd VSUX

                                                                                                            VSU

                                                                                                            )min(1 dnss

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Full Representation

                                                                                                            =

                                                                                                            Graphics Display Assumes

                                                                                                            ndX ddU ndS nn

                                                                                                            tV

                                                                                                            nd

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Full Representation

                                                                                                            =

                                                                                                            Full Rank Basis Matrix

                                                                                                            ndX ddU ndS nn

                                                                                                            tV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Full Representation

                                                                                                            =

                                                                                                            Full Rank Basis Matrix

                                                                                                            All 0s in Bottom

                                                                                                            ndX ddU ndS nn

                                                                                                            tV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Reduced Representation

                                                                                                            =

                                                                                                            These Columns Get 0ed Out

                                                                                                            ndX ddU nnS nn

                                                                                                            tV

                                                                                                            nnd 0

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Reduced Representation

                                                                                                            =ndX ndU nnS nn

                                                                                                            tV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Reduced Representation

                                                                                                            =

                                                                                                            Also Some of These May be 0

                                                                                                            ndX ndU nnS nn

                                                                                                            tV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Compact Representation

                                                                                                            = ndX rdU

                                                                                                            rrS nrtV

                                                                                                            0

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Compact Representation

                                                                                                            =

                                                                                                            These Get 0ed Out

                                                                                                            ndX rdU

                                                                                                            rrS nrtV

                                                                                                            0

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            SVD Compact Representation

                                                                                                            = ndX rdU

                                                                                                            rrS nrtV

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition

                                                                                                            For a (Symmetric) Square Matrix

                                                                                                            Find

                                                                                                            ddX

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition

                                                                                                            For a (Symmetric) Square Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            Called Eigenvalues

                                                                                                            Convenient Ordering

                                                                                                            ddX

                                                                                                            d

                                                                                                            D

                                                                                                            0

                                                                                                            01

                                                                                                            n 1

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition

                                                                                                            For a (Symmetric) Square Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            And an Orthonormal Matrix

                                                                                                            (ie )

                                                                                                            ddX

                                                                                                            d

                                                                                                            D

                                                                                                            0

                                                                                                            01

                                                                                                            ddB

                                                                                                            ddtt IBBBB

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition

                                                                                                            For a (Symmetric) Square Matrix

                                                                                                            Find a Diagonal Matrix

                                                                                                            And an Orthonormal Matrix

                                                                                                            (ie )

                                                                                                            So that ie

                                                                                                            ddX

                                                                                                            d

                                                                                                            D

                                                                                                            0

                                                                                                            01

                                                                                                            ddB

                                                                                                            ddtt IBBBB

                                                                                                            DBBX tBDBX

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                            (looks similar)

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                            Complex (uses )

                                                                                                            VU

                                                                                                            1i

                                                                                                            Review of Linear Algebra (Cont)

                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                            VU

                                                                                                            X

                                                                                                            BVU

                                                                                                            1i

                                                                                                            • Participant Presentations
                                                                                                            • PCA to find clusters
                                                                                                            • Statistical Smoothing
                                                                                                            • Kernel Density Estimation
                                                                                                            • Scatterplot Smoothing
                                                                                                            • Statistical Smoothing (2)
                                                                                                            • SiZer Background
                                                                                                            • SiZer Background (2)
                                                                                                            • SiZer Background (3)
                                                                                                            • SiZer Background (4)
                                                                                                            • SiZer Background (5)
                                                                                                            • SiZer Overview
                                                                                                            • PCA to find clusters (2)
                                                                                                            • PCA to find clusters (3)
                                                                                                            • PCA to find clusters (4)
                                                                                                            • PCA to find clusters (5)
                                                                                                            • PCA to find clusters (6)
                                                                                                            • PCA to find clusters (7)
                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                            • Frequency 2 Analysis
                                                                                                            • Frequency 2 Analysis (2)
                                                                                                            • Frequency 2 Analysis (3)
                                                                                                            • Yeast Cell Cycle
                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                            • Reclassification of Major Genes
                                                                                                            • Compare to Previous Classifrsquon
                                                                                                            • New Subpopulation View
                                                                                                            • New Subpopulation View (2)
                                                                                                            • Detailed Look at PCA
                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                            • An Interesting Historical Note
                                                                                                            • An Interesting Historical Note (2)
                                                                                                            • Detailed Look at PCA (2)
                                                                                                            • Detailed Look at PCA (3)
                                                                                                            • Review of Linear Algebra
                                                                                                            • Review of Linear Algebra (2)
                                                                                                            • Review of Linear Algebra (3)
                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Then linear combo is a matrix multiplicatrsquon

                                                                                                              where

                                                                                                              Check sizes

                                                                                                              n

                                                                                                              iii aBva

                                                                                                              1

                                                                                                              na

                                                                                                              a

                                                                                                              a 1

                                                                                                              )1()(1 nndd

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                              For matrices

                                                                                                              Define the Matrix Product

                                                                                                              mkk

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              nmm

                                                                                                              n

                                                                                                              bb

                                                                                                              bb

                                                                                                              B

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iniik

                                                                                                              m

                                                                                                              iiik

                                                                                                              m

                                                                                                              inii

                                                                                                              m

                                                                                                              iii

                                                                                                              baba

                                                                                                              baba

                                                                                                              AB

                                                                                                              1

                                                                                                              11

                                                                                                              11

                                                                                                              111

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                              For matrices

                                                                                                              Define the Matrix Product

                                                                                                              (Inner Products of Rows With Columns )

                                                                                                              mkk

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              nmm

                                                                                                              n

                                                                                                              bb

                                                                                                              bb

                                                                                                              B

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iniik

                                                                                                              m

                                                                                                              iiik

                                                                                                              m

                                                                                                              inii

                                                                                                              m

                                                                                                              iii

                                                                                                              baba

                                                                                                              baba

                                                                                                              AB

                                                                                                              1

                                                                                                              11

                                                                                                              11

                                                                                                              111

                                                                                                              A B

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                              For matrices

                                                                                                              Define the Matrix Product

                                                                                                              (Inner Products of Rows With Columns )

                                                                                                              (Composition of Linear Transformations)

                                                                                                              mkk

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              nmm

                                                                                                              n

                                                                                                              bb

                                                                                                              bb

                                                                                                              B

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iniik

                                                                                                              m

                                                                                                              iiik

                                                                                                              m

                                                                                                              inii

                                                                                                              m

                                                                                                              iii

                                                                                                              baba

                                                                                                              baba

                                                                                                              AB

                                                                                                              1

                                                                                                              11

                                                                                                              11

                                                                                                              111

                                                                                                              A B

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                              For matrices

                                                                                                              Define the Matrix Product

                                                                                                              (Inner Products of Rows With Columns )

                                                                                                              (Composition of Linear Transformations)

                                                                                                              Often Useful to Check Sizes

                                                                                                              mkk

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              nmm

                                                                                                              n

                                                                                                              bb

                                                                                                              bb

                                                                                                              B

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iniik

                                                                                                              m

                                                                                                              iiik

                                                                                                              m

                                                                                                              inii

                                                                                                              m

                                                                                                              iii

                                                                                                              baba

                                                                                                              baba

                                                                                                              AB

                                                                                                              1

                                                                                                              11

                                                                                                              11

                                                                                                              111

                                                                                                              nmmknk

                                                                                                              A B

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                              For matrices

                                                                                                              Define the Matrix Product

                                                                                                              (Inner Products of Rows With Columns )

                                                                                                              (Composition of Linear Transformations)

                                                                                                              Often Useful to Check Sizes

                                                                                                              mkk

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              nmm

                                                                                                              n

                                                                                                              bb

                                                                                                              bb

                                                                                                              B

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iniik

                                                                                                              m

                                                                                                              iiik

                                                                                                              m

                                                                                                              inii

                                                                                                              m

                                                                                                              iii

                                                                                                              baba

                                                                                                              baba

                                                                                                              AB

                                                                                                              1

                                                                                                              11

                                                                                                              11

                                                                                                              111

                                                                                                              nmmknk

                                                                                                              A B

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Matrix Trace

                                                                                                              bull For a Square Matrix

                                                                                                              bull Define

                                                                                                              mmm

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iiiaAtr

                                                                                                              1)(

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Matrix Trace

                                                                                                              bull For a Square Matrix

                                                                                                              bull Define

                                                                                                              bull Trace Commutes with Matrix Multiplication

                                                                                                              mmm

                                                                                                              m

                                                                                                              aa

                                                                                                              aa

                                                                                                              A

                                                                                                              1

                                                                                                              111

                                                                                                              m

                                                                                                              iiiaAtr

                                                                                                              1)(

                                                                                                              BAtrABtr

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                              bull (Use Basis Above)

                                                                                                              bull eg dim of a line is 1

                                                                                                              bull eg dim of a plane is 2

                                                                                                              dd dim

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                              bull (Use Basis Above)

                                                                                                              bull eg dim of a line is 1

                                                                                                              bull eg dim of a plane is 2

                                                                                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                              (in Statistical Uses eg ANOVA)

                                                                                                              dd dim

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Norm of a Vector

                                                                                                              bull in d 21

                                                                                                              21

                                                                                                              1

                                                                                                              2 xxxx td

                                                                                                              jj

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Norm of a Vector

                                                                                                              bull in

                                                                                                              bull Idea length of the vector

                                                                                                              d 2121

                                                                                                              1

                                                                                                              2 xxxx td

                                                                                                              jj

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Norm of a Vector

                                                                                                              bull in

                                                                                                              bull Idea length of the vector

                                                                                                              bull Note strange properties for high

                                                                                                              eg ldquolength of diagonal of unit cuberdquo =

                                                                                                              d 2121

                                                                                                              1

                                                                                                              2 xxxx td

                                                                                                              jj

                                                                                                              d

                                                                                                              d

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Norm of a Vector (cont)

                                                                                                              bull Length Normalized Vector

                                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                                              amp is a Direction Vector)

                                                                                                              x

                                                                                                              x

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Norm of a Vector (cont)

                                                                                                              bull Length Normalized Vector

                                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                                              amp is a Direction Vector)

                                                                                                              bull Define Distance as

                                                                                                              x

                                                                                                              x

                                                                                                              yxyxyxyxd t

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Inner (Dot Scalar) Product

                                                                                                              bull for Vectors and

                                                                                                              yxyxyx td

                                                                                                              jjj

                                                                                                              1

                                                                                                              x y

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Inner (Dot Scalar) Product

                                                                                                              bull for Vectors and

                                                                                                              bull Related to Norm via

                                                                                                              yxyxyx td

                                                                                                              jjj

                                                                                                              1

                                                                                                              x y

                                                                                                              21 xxx

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                              yyxx

                                                                                                              yx

                                                                                                              yx

                                                                                                              yxyxangle

                                                                                                              tt

                                                                                                              t

                                                                                                              11 cos

                                                                                                              cos

                                                                                                              x y

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                              if and only if

                                                                                                              yyxx

                                                                                                              yx

                                                                                                              yx

                                                                                                              yxyxangle

                                                                                                              tt

                                                                                                              t

                                                                                                              11 cos

                                                                                                              cos

                                                                                                              x y

                                                                                                              yx 0 yx

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis

                                                                                                              bull All Orthogonal to each other

                                                                                                              ie for

                                                                                                              bull All have Length 1

                                                                                                              ie for

                                                                                                              nvv 1

                                                                                                              1 ii vv

                                                                                                              0 ii vv ii

                                                                                                              ni 1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis (cont)

                                                                                                              bull Spectral Representation

                                                                                                              where

                                                                                                              nvv 1

                                                                                                              n

                                                                                                              iii vax

                                                                                                              1

                                                                                                              ii vxa

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis (cont)

                                                                                                              bull Spectral Representation

                                                                                                              where

                                                                                                              (Coefficient is Inner Product Cool Notation)

                                                                                                              nvv 1

                                                                                                              n

                                                                                                              iii vax

                                                                                                              1

                                                                                                              ii vxa

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis (cont)

                                                                                                              bull Spectral Representation

                                                                                                              where

                                                                                                              Check

                                                                                                              nvv 1

                                                                                                              n

                                                                                                              iii vax

                                                                                                              1

                                                                                                              ii vxa

                                                                                                              iii

                                                                                                              n

                                                                                                              iii

                                                                                                              n

                                                                                                              iiii avvavvavx

                                                                                                              1

                                                                                                              1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis (cont)

                                                                                                              bull Spectral Representation

                                                                                                              where

                                                                                                              Check

                                                                                                              bull Matrix Notation where ie

                                                                                                              For the Basis Matrix

                                                                                                              nvv 1

                                                                                                              n

                                                                                                              iii vax

                                                                                                              1

                                                                                                              ii vxa

                                                                                                              iii

                                                                                                              n

                                                                                                              iii

                                                                                                              n

                                                                                                              iiii avvavvavx

                                                                                                              1

                                                                                                              1

                                                                                                              aBx Bxa tt xBa t

                                                                                                              nvvB 1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Orthonormal Basis (cont)

                                                                                                              bull Spectral Representation

                                                                                                              where

                                                                                                              Check

                                                                                                              bull Matrix Notation where ie

                                                                                                              is called transform of

                                                                                                              (eg Fourier or Wavelet)

                                                                                                              nvv 1

                                                                                                              n

                                                                                                              iii vax

                                                                                                              1

                                                                                                              ii vxa

                                                                                                              iii

                                                                                                              n

                                                                                                              iii

                                                                                                              n

                                                                                                              iiii avvavvavx

                                                                                                              1

                                                                                                              1

                                                                                                              aBx Bxa tt xBa t

                                                                                                              xa

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Parseval identity for

                                                                                                              in subsp genrsquod by o n basis

                                                                                                              x

                                                                                                              nvv 1

                                                                                                              2

                                                                                                              1

                                                                                                              22

                                                                                                              1

                                                                                                              2 aavxx

                                                                                                              n

                                                                                                              ii

                                                                                                              n

                                                                                                              ii

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Parseval identity for

                                                                                                              in subsp genrsquod by o n basis

                                                                                                              bull Pythagorean theorem

                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                              bull ANOVA - sums of squares

                                                                                                              x

                                                                                                              nvv 1

                                                                                                              2

                                                                                                              1

                                                                                                              22

                                                                                                              1

                                                                                                              2 aavxx

                                                                                                              n

                                                                                                              ii

                                                                                                              n

                                                                                                              ii

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Parseval identity for

                                                                                                              in subsp genrsquod by o n basis

                                                                                                              bull Pythagorean theorem

                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                              bull ANOVA - sums of squares

                                                                                                              bull Transform has same length as

                                                                                                              ie ldquorotation in rdquo

                                                                                                              x

                                                                                                              nvv 1

                                                                                                              2

                                                                                                              1

                                                                                                              22

                                                                                                              1

                                                                                                              2 aavxx

                                                                                                              n

                                                                                                              ii

                                                                                                              n

                                                                                                              ii

                                                                                                              a xd

                                                                                                              Projection of a Vector onto a Subspace

                                                                                                              bull Idea Member of that is Closest to

                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                              xV

                                                                                                              V

                                                                                                              Projection of a Vector onto a Subspace

                                                                                                              bull Idea Member of that is Closest to

                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                              bull Find that Solves

                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                              xV

                                                                                                              V

                                                                                                              VxPV vxVv

                                                                                                              min

                                                                                                              Projection of a Vector onto a Subspace

                                                                                                              bull Idea Member of that is Closest to

                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                              bull Find that Solves

                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                              bull For Inner Product (Hilbert) Space

                                                                                                              Exists and is Unique

                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                              xV

                                                                                                              V

                                                                                                              VxPV vxVv

                                                                                                              min

                                                                                                              xPV

                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                              bull General Solution in for Basis Matrix

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              d VB

                                                                                                              xBBBBxP tVV

                                                                                                              tVVV

                                                                                                              1

                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                              bull General Solution in for Basis Matrix

                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                              (thus projection is another linear operation)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              d VB

                                                                                                              xBBBBxP tVV

                                                                                                              tVVV

                                                                                                              1

                                                                                                              tVV

                                                                                                              tVVV BBBBP

                                                                                                              1

                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                              bull General Solution in for Basis Matrix

                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                              (thus projection is another linear operation)

                                                                                                              (note same operation underlies least squares)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              d VB

                                                                                                              xBBBBxP tVV

                                                                                                              tVVV

                                                                                                              1

                                                                                                              tVV

                                                                                                              tVVV BBBBP

                                                                                                              1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Projection using Orthonormal Basis

                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                              nnVtV IBB

                                                                                                              10

                                                                                                              01

                                                                                                              1

                                                                                                              111

                                                                                                              1

                                                                                                              1

                                                                                                              nnn

                                                                                                              n

                                                                                                              ntn

                                                                                                              t

                                                                                                              vvvv

                                                                                                              vvvv

                                                                                                              vv

                                                                                                              v

                                                                                                              v

                                                                                                              nvv 1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Projection using Orthonormal Basis

                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                              bull So =

                                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                              (Recall Right Multrsquon)

                                                                                                              nnVtV IBB

                                                                                                              10

                                                                                                              01

                                                                                                              1

                                                                                                              111

                                                                                                              1

                                                                                                              1

                                                                                                              nnn

                                                                                                              n

                                                                                                              ntn

                                                                                                              t

                                                                                                              vvvv

                                                                                                              vvvv

                                                                                                              vv

                                                                                                              v

                                                                                                              v

                                                                                                              xBBxP tVVV

                                                                                                              x V

                                                                                                              nvv 1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                              bull For Orthogonal Complement

                                                                                                              and

                                                                                                              V

                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                              bull For Orthogonal Complement

                                                                                                              and

                                                                                                              bull Parseval Inequality

                                                                                                              V

                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                              2

                                                                                                              1

                                                                                                              22

                                                                                                              1

                                                                                                              22 aavxxxP

                                                                                                              n

                                                                                                              ii

                                                                                                              n

                                                                                                              iiV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              (Real) Unitary Matrices with

                                                                                                              bull Orthonormal Basis Matrix

                                                                                                              (So All of Above Applies)

                                                                                                              ddU IUU t

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              (Real) Unitary Matrices with

                                                                                                              bull Orthonormal Basis Matrix

                                                                                                              (So All of Above Applies)

                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                              ddU IUU t

                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                              i ii 2

                                                                                                              1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              (Real) Unitary Matrices with

                                                                                                              bull Orthonormal Basis Matrix

                                                                                                              (So All of Above Applies)

                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                              ddU IUU t

                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                              i ii 2

                                                                                                              1

                                                                                                              U

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Singular Value Decomposition (SVD)

                                                                                                              For a Matrix

                                                                                                              Find

                                                                                                              ndX

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Singular Value Decomposition (SVD)

                                                                                                              For a Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              with Entries

                                                                                                              called Singular Values

                                                                                                              ndX

                                                                                                              ndS

                                                                                                              )min(1 ndss

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Singular Value Decomposition (SVD)

                                                                                                              For a Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              with Entries

                                                                                                              called Singular Values

                                                                                                              And Unitary (Rotation) Matrices

                                                                                                              (recall )

                                                                                                              ndX

                                                                                                              ndS

                                                                                                              )min(1 ndss

                                                                                                              ddU nnV

                                                                                                              IVVUU tt

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Singular Value Decomposition (SVD)

                                                                                                              For a Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              with Entries

                                                                                                              called Singular Values

                                                                                                              And Unitary (Rotation) Matrices

                                                                                                              (recall )

                                                                                                              So That

                                                                                                              ndX

                                                                                                              ndS

                                                                                                              )min(1 ndss

                                                                                                              ddU nnV

                                                                                                              IVVUU tt tUSVX

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                              vVSUvVSUvX tt

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                              bull First rotate

                                                                                                              X

                                                                                                              vVSUvVSUvX tt

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                              bull First rotate

                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                              X

                                                                                                              vVSUvVSUvX tt

                                                                                                              is

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                              bull First rotate

                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                              bull Third rotate again

                                                                                                              X

                                                                                                              vVSUvVSUvX tt

                                                                                                              is

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                              bull First rotate

                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                              bull Third rotate again

                                                                                                              bull ie have diagonalized the transformation

                                                                                                              X

                                                                                                              vVSUvVSUvX tt

                                                                                                              is

                                                                                                              SVD Compact Representation

                                                                                                              Useful Labeling

                                                                                                              Singular Values in Increasing Order

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              )min(1 dnss

                                                                                                              SVD Compact Representation

                                                                                                              Useful Labeling

                                                                                                              Singular Values in Increasing Order

                                                                                                              Note singular values = 0 can be omitted

                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              )min(1 dnss

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              r

                                                                                                              SVD Compact Representation

                                                                                                              Useful Labeling

                                                                                                              Singular Values in Increasing Order

                                                                                                              Note singular values = 0 can be omitted

                                                                                                              Let = of positive singular values

                                                                                                              )min(1 dnss

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              r

                                                                                                              SVD Compact Representation

                                                                                                              Useful Labeling

                                                                                                              Singular Values in Increasing Order

                                                                                                              Note singular values = 0 can be omitted

                                                                                                              Let = of positive singular values

                                                                                                              Then

                                                                                                              Where are truncations of

                                                                                                              trnrrrd VSUX

                                                                                                              VSU

                                                                                                              )min(1 dnss

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Full Representation

                                                                                                              =

                                                                                                              Graphics Display Assumes

                                                                                                              ndX ddU ndS nn

                                                                                                              tV

                                                                                                              nd

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Full Representation

                                                                                                              =

                                                                                                              Full Rank Basis Matrix

                                                                                                              ndX ddU ndS nn

                                                                                                              tV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Full Representation

                                                                                                              =

                                                                                                              Full Rank Basis Matrix

                                                                                                              All 0s in Bottom

                                                                                                              ndX ddU ndS nn

                                                                                                              tV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Reduced Representation

                                                                                                              =

                                                                                                              These Columns Get 0ed Out

                                                                                                              ndX ddU nnS nn

                                                                                                              tV

                                                                                                              nnd 0

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Reduced Representation

                                                                                                              =ndX ndU nnS nn

                                                                                                              tV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Reduced Representation

                                                                                                              =

                                                                                                              Also Some of These May be 0

                                                                                                              ndX ndU nnS nn

                                                                                                              tV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Compact Representation

                                                                                                              = ndX rdU

                                                                                                              rrS nrtV

                                                                                                              0

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Compact Representation

                                                                                                              =

                                                                                                              These Get 0ed Out

                                                                                                              ndX rdU

                                                                                                              rrS nrtV

                                                                                                              0

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              SVD Compact Representation

                                                                                                              = ndX rdU

                                                                                                              rrS nrtV

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition

                                                                                                              For a (Symmetric) Square Matrix

                                                                                                              Find

                                                                                                              ddX

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition

                                                                                                              For a (Symmetric) Square Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              Called Eigenvalues

                                                                                                              Convenient Ordering

                                                                                                              ddX

                                                                                                              d

                                                                                                              D

                                                                                                              0

                                                                                                              01

                                                                                                              n 1

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition

                                                                                                              For a (Symmetric) Square Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              And an Orthonormal Matrix

                                                                                                              (ie )

                                                                                                              ddX

                                                                                                              d

                                                                                                              D

                                                                                                              0

                                                                                                              01

                                                                                                              ddB

                                                                                                              ddtt IBBBB

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition

                                                                                                              For a (Symmetric) Square Matrix

                                                                                                              Find a Diagonal Matrix

                                                                                                              And an Orthonormal Matrix

                                                                                                              (ie )

                                                                                                              So that ie

                                                                                                              ddX

                                                                                                              d

                                                                                                              D

                                                                                                              0

                                                                                                              01

                                                                                                              ddB

                                                                                                              ddtt IBBBB

                                                                                                              DBBX tBDBX

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                              (looks similar)

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                              Complex (uses )

                                                                                                              VU

                                                                                                              1i

                                                                                                              Review of Linear Algebra (Cont)

                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                              VU

                                                                                                              X

                                                                                                              BVU

                                                                                                              1i

                                                                                                              • Participant Presentations
                                                                                                              • PCA to find clusters
                                                                                                              • Statistical Smoothing
                                                                                                              • Kernel Density Estimation
                                                                                                              • Scatterplot Smoothing
                                                                                                              • Statistical Smoothing (2)
                                                                                                              • SiZer Background
                                                                                                              • SiZer Background (2)
                                                                                                              • SiZer Background (3)
                                                                                                              • SiZer Background (4)
                                                                                                              • SiZer Background (5)
                                                                                                              • SiZer Overview
                                                                                                              • PCA to find clusters (2)
                                                                                                              • PCA to find clusters (3)
                                                                                                              • PCA to find clusters (4)
                                                                                                              • PCA to find clusters (5)
                                                                                                              • PCA to find clusters (6)
                                                                                                              • PCA to find clusters (7)
                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                              • Frequency 2 Analysis
                                                                                                              • Frequency 2 Analysis (2)
                                                                                                              • Frequency 2 Analysis (3)
                                                                                                              • Yeast Cell Cycle
                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                              • Reclassification of Major Genes
                                                                                                              • Compare to Previous Classifrsquon
                                                                                                              • New Subpopulation View
                                                                                                              • New Subpopulation View (2)
                                                                                                              • Detailed Look at PCA
                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                              • An Interesting Historical Note
                                                                                                              • An Interesting Historical Note (2)
                                                                                                              • Detailed Look at PCA (2)
                                                                                                              • Detailed Look at PCA (3)
                                                                                                              • Review of Linear Algebra
                                                                                                              • Review of Linear Algebra (2)
                                                                                                              • Review of Linear Algebra (3)
                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                For matrices

                                                                                                                Define the Matrix Product

                                                                                                                mkk

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                nmm

                                                                                                                n

                                                                                                                bb

                                                                                                                bb

                                                                                                                B

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iniik

                                                                                                                m

                                                                                                                iiik

                                                                                                                m

                                                                                                                inii

                                                                                                                m

                                                                                                                iii

                                                                                                                baba

                                                                                                                baba

                                                                                                                AB

                                                                                                                1

                                                                                                                11

                                                                                                                11

                                                                                                                111

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                For matrices

                                                                                                                Define the Matrix Product

                                                                                                                (Inner Products of Rows With Columns )

                                                                                                                mkk

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                nmm

                                                                                                                n

                                                                                                                bb

                                                                                                                bb

                                                                                                                B

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iniik

                                                                                                                m

                                                                                                                iiik

                                                                                                                m

                                                                                                                inii

                                                                                                                m

                                                                                                                iii

                                                                                                                baba

                                                                                                                baba

                                                                                                                AB

                                                                                                                1

                                                                                                                11

                                                                                                                11

                                                                                                                111

                                                                                                                A B

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                For matrices

                                                                                                                Define the Matrix Product

                                                                                                                (Inner Products of Rows With Columns )

                                                                                                                (Composition of Linear Transformations)

                                                                                                                mkk

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                nmm

                                                                                                                n

                                                                                                                bb

                                                                                                                bb

                                                                                                                B

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iniik

                                                                                                                m

                                                                                                                iiik

                                                                                                                m

                                                                                                                inii

                                                                                                                m

                                                                                                                iii

                                                                                                                baba

                                                                                                                baba

                                                                                                                AB

                                                                                                                1

                                                                                                                11

                                                                                                                11

                                                                                                                111

                                                                                                                A B

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                For matrices

                                                                                                                Define the Matrix Product

                                                                                                                (Inner Products of Rows With Columns )

                                                                                                                (Composition of Linear Transformations)

                                                                                                                Often Useful to Check Sizes

                                                                                                                mkk

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                nmm

                                                                                                                n

                                                                                                                bb

                                                                                                                bb

                                                                                                                B

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iniik

                                                                                                                m

                                                                                                                iiik

                                                                                                                m

                                                                                                                inii

                                                                                                                m

                                                                                                                iii

                                                                                                                baba

                                                                                                                baba

                                                                                                                AB

                                                                                                                1

                                                                                                                11

                                                                                                                11

                                                                                                                111

                                                                                                                nmmknk

                                                                                                                A B

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                For matrices

                                                                                                                Define the Matrix Product

                                                                                                                (Inner Products of Rows With Columns )

                                                                                                                (Composition of Linear Transformations)

                                                                                                                Often Useful to Check Sizes

                                                                                                                mkk

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                nmm

                                                                                                                n

                                                                                                                bb

                                                                                                                bb

                                                                                                                B

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iniik

                                                                                                                m

                                                                                                                iiik

                                                                                                                m

                                                                                                                inii

                                                                                                                m

                                                                                                                iii

                                                                                                                baba

                                                                                                                baba

                                                                                                                AB

                                                                                                                1

                                                                                                                11

                                                                                                                11

                                                                                                                111

                                                                                                                nmmknk

                                                                                                                A B

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Matrix Trace

                                                                                                                bull For a Square Matrix

                                                                                                                bull Define

                                                                                                                mmm

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iiiaAtr

                                                                                                                1)(

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Matrix Trace

                                                                                                                bull For a Square Matrix

                                                                                                                bull Define

                                                                                                                bull Trace Commutes with Matrix Multiplication

                                                                                                                mmm

                                                                                                                m

                                                                                                                aa

                                                                                                                aa

                                                                                                                A

                                                                                                                1

                                                                                                                111

                                                                                                                m

                                                                                                                iiiaAtr

                                                                                                                1)(

                                                                                                                BAtrABtr

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                                bull (Use Basis Above)

                                                                                                                bull eg dim of a line is 1

                                                                                                                bull eg dim of a plane is 2

                                                                                                                dd dim

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                                bull (Use Basis Above)

                                                                                                                bull eg dim of a line is 1

                                                                                                                bull eg dim of a plane is 2

                                                                                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                (in Statistical Uses eg ANOVA)

                                                                                                                dd dim

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Norm of a Vector

                                                                                                                bull in d 21

                                                                                                                21

                                                                                                                1

                                                                                                                2 xxxx td

                                                                                                                jj

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Norm of a Vector

                                                                                                                bull in

                                                                                                                bull Idea length of the vector

                                                                                                                d 2121

                                                                                                                1

                                                                                                                2 xxxx td

                                                                                                                jj

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Norm of a Vector

                                                                                                                bull in

                                                                                                                bull Idea length of the vector

                                                                                                                bull Note strange properties for high

                                                                                                                eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                d 2121

                                                                                                                1

                                                                                                                2 xxxx td

                                                                                                                jj

                                                                                                                d

                                                                                                                d

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Norm of a Vector (cont)

                                                                                                                bull Length Normalized Vector

                                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                                amp is a Direction Vector)

                                                                                                                x

                                                                                                                x

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Norm of a Vector (cont)

                                                                                                                bull Length Normalized Vector

                                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                                amp is a Direction Vector)

                                                                                                                bull Define Distance as

                                                                                                                x

                                                                                                                x

                                                                                                                yxyxyxyxd t

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Inner (Dot Scalar) Product

                                                                                                                bull for Vectors and

                                                                                                                yxyxyx td

                                                                                                                jjj

                                                                                                                1

                                                                                                                x y

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Inner (Dot Scalar) Product

                                                                                                                bull for Vectors and

                                                                                                                bull Related to Norm via

                                                                                                                yxyxyx td

                                                                                                                jjj

                                                                                                                1

                                                                                                                x y

                                                                                                                21 xxx

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                yyxx

                                                                                                                yx

                                                                                                                yx

                                                                                                                yxyxangle

                                                                                                                tt

                                                                                                                t

                                                                                                                11 cos

                                                                                                                cos

                                                                                                                x y

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                if and only if

                                                                                                                yyxx

                                                                                                                yx

                                                                                                                yx

                                                                                                                yxyxangle

                                                                                                                tt

                                                                                                                t

                                                                                                                11 cos

                                                                                                                cos

                                                                                                                x y

                                                                                                                yx 0 yx

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis

                                                                                                                bull All Orthogonal to each other

                                                                                                                ie for

                                                                                                                bull All have Length 1

                                                                                                                ie for

                                                                                                                nvv 1

                                                                                                                1 ii vv

                                                                                                                0 ii vv ii

                                                                                                                ni 1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis (cont)

                                                                                                                bull Spectral Representation

                                                                                                                where

                                                                                                                nvv 1

                                                                                                                n

                                                                                                                iii vax

                                                                                                                1

                                                                                                                ii vxa

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis (cont)

                                                                                                                bull Spectral Representation

                                                                                                                where

                                                                                                                (Coefficient is Inner Product Cool Notation)

                                                                                                                nvv 1

                                                                                                                n

                                                                                                                iii vax

                                                                                                                1

                                                                                                                ii vxa

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis (cont)

                                                                                                                bull Spectral Representation

                                                                                                                where

                                                                                                                Check

                                                                                                                nvv 1

                                                                                                                n

                                                                                                                iii vax

                                                                                                                1

                                                                                                                ii vxa

                                                                                                                iii

                                                                                                                n

                                                                                                                iii

                                                                                                                n

                                                                                                                iiii avvavvavx

                                                                                                                1

                                                                                                                1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis (cont)

                                                                                                                bull Spectral Representation

                                                                                                                where

                                                                                                                Check

                                                                                                                bull Matrix Notation where ie

                                                                                                                For the Basis Matrix

                                                                                                                nvv 1

                                                                                                                n

                                                                                                                iii vax

                                                                                                                1

                                                                                                                ii vxa

                                                                                                                iii

                                                                                                                n

                                                                                                                iii

                                                                                                                n

                                                                                                                iiii avvavvavx

                                                                                                                1

                                                                                                                1

                                                                                                                aBx Bxa tt xBa t

                                                                                                                nvvB 1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Orthonormal Basis (cont)

                                                                                                                bull Spectral Representation

                                                                                                                where

                                                                                                                Check

                                                                                                                bull Matrix Notation where ie

                                                                                                                is called transform of

                                                                                                                (eg Fourier or Wavelet)

                                                                                                                nvv 1

                                                                                                                n

                                                                                                                iii vax

                                                                                                                1

                                                                                                                ii vxa

                                                                                                                iii

                                                                                                                n

                                                                                                                iii

                                                                                                                n

                                                                                                                iiii avvavvavx

                                                                                                                1

                                                                                                                1

                                                                                                                aBx Bxa tt xBa t

                                                                                                                xa

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Parseval identity for

                                                                                                                in subsp genrsquod by o n basis

                                                                                                                x

                                                                                                                nvv 1

                                                                                                                2

                                                                                                                1

                                                                                                                22

                                                                                                                1

                                                                                                                2 aavxx

                                                                                                                n

                                                                                                                ii

                                                                                                                n

                                                                                                                ii

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Parseval identity for

                                                                                                                in subsp genrsquod by o n basis

                                                                                                                bull Pythagorean theorem

                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                bull ANOVA - sums of squares

                                                                                                                x

                                                                                                                nvv 1

                                                                                                                2

                                                                                                                1

                                                                                                                22

                                                                                                                1

                                                                                                                2 aavxx

                                                                                                                n

                                                                                                                ii

                                                                                                                n

                                                                                                                ii

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Parseval identity for

                                                                                                                in subsp genrsquod by o n basis

                                                                                                                bull Pythagorean theorem

                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                bull ANOVA - sums of squares

                                                                                                                bull Transform has same length as

                                                                                                                ie ldquorotation in rdquo

                                                                                                                x

                                                                                                                nvv 1

                                                                                                                2

                                                                                                                1

                                                                                                                22

                                                                                                                1

                                                                                                                2 aavxx

                                                                                                                n

                                                                                                                ii

                                                                                                                n

                                                                                                                ii

                                                                                                                a xd

                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                bull Idea Member of that is Closest to

                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                xV

                                                                                                                V

                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                bull Idea Member of that is Closest to

                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                bull Find that Solves

                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                xV

                                                                                                                V

                                                                                                                VxPV vxVv

                                                                                                                min

                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                bull Idea Member of that is Closest to

                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                bull Find that Solves

                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                bull For Inner Product (Hilbert) Space

                                                                                                                Exists and is Unique

                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                xV

                                                                                                                V

                                                                                                                VxPV vxVv

                                                                                                                min

                                                                                                                xPV

                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                d VB

                                                                                                                xBBBBxP tVV

                                                                                                                tVVV

                                                                                                                1

                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                (thus projection is another linear operation)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                d VB

                                                                                                                xBBBBxP tVV

                                                                                                                tVVV

                                                                                                                1

                                                                                                                tVV

                                                                                                                tVVV BBBBP

                                                                                                                1

                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                (thus projection is another linear operation)

                                                                                                                (note same operation underlies least squares)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                d VB

                                                                                                                xBBBBxP tVV

                                                                                                                tVVV

                                                                                                                1

                                                                                                                tVV

                                                                                                                tVVV BBBBP

                                                                                                                1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Projection using Orthonormal Basis

                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                nnVtV IBB

                                                                                                                10

                                                                                                                01

                                                                                                                1

                                                                                                                111

                                                                                                                1

                                                                                                                1

                                                                                                                nnn

                                                                                                                n

                                                                                                                ntn

                                                                                                                t

                                                                                                                vvvv

                                                                                                                vvvv

                                                                                                                vv

                                                                                                                v

                                                                                                                v

                                                                                                                nvv 1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Projection using Orthonormal Basis

                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                bull So =

                                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                (Recall Right Multrsquon)

                                                                                                                nnVtV IBB

                                                                                                                10

                                                                                                                01

                                                                                                                1

                                                                                                                111

                                                                                                                1

                                                                                                                1

                                                                                                                nnn

                                                                                                                n

                                                                                                                ntn

                                                                                                                t

                                                                                                                vvvv

                                                                                                                vvvv

                                                                                                                vv

                                                                                                                v

                                                                                                                v

                                                                                                                xBBxP tVVV

                                                                                                                x V

                                                                                                                nvv 1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                bull For Orthogonal Complement

                                                                                                                and

                                                                                                                V

                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                bull For Orthogonal Complement

                                                                                                                and

                                                                                                                bull Parseval Inequality

                                                                                                                V

                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                2

                                                                                                                1

                                                                                                                22

                                                                                                                1

                                                                                                                22 aavxxxP

                                                                                                                n

                                                                                                                ii

                                                                                                                n

                                                                                                                iiV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                (Real) Unitary Matrices with

                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                (So All of Above Applies)

                                                                                                                ddU IUU t

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                (Real) Unitary Matrices with

                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                (So All of Above Applies)

                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                ddU IUU t

                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                i ii 2

                                                                                                                1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                (Real) Unitary Matrices with

                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                (So All of Above Applies)

                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                ddU IUU t

                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                i ii 2

                                                                                                                1

                                                                                                                U

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                For a Matrix

                                                                                                                Find

                                                                                                                ndX

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                For a Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                with Entries

                                                                                                                called Singular Values

                                                                                                                ndX

                                                                                                                ndS

                                                                                                                )min(1 ndss

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                For a Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                with Entries

                                                                                                                called Singular Values

                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                (recall )

                                                                                                                ndX

                                                                                                                ndS

                                                                                                                )min(1 ndss

                                                                                                                ddU nnV

                                                                                                                IVVUU tt

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                For a Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                with Entries

                                                                                                                called Singular Values

                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                (recall )

                                                                                                                So That

                                                                                                                ndX

                                                                                                                ndS

                                                                                                                )min(1 ndss

                                                                                                                ddU nnV

                                                                                                                IVVUU tt tUSVX

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                vVSUvVSUvX tt

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                bull First rotate

                                                                                                                X

                                                                                                                vVSUvVSUvX tt

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                bull First rotate

                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                X

                                                                                                                vVSUvVSUvX tt

                                                                                                                is

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                bull First rotate

                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                bull Third rotate again

                                                                                                                X

                                                                                                                vVSUvVSUvX tt

                                                                                                                is

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                bull First rotate

                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                bull Third rotate again

                                                                                                                bull ie have diagonalized the transformation

                                                                                                                X

                                                                                                                vVSUvVSUvX tt

                                                                                                                is

                                                                                                                SVD Compact Representation

                                                                                                                Useful Labeling

                                                                                                                Singular Values in Increasing Order

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                )min(1 dnss

                                                                                                                SVD Compact Representation

                                                                                                                Useful Labeling

                                                                                                                Singular Values in Increasing Order

                                                                                                                Note singular values = 0 can be omitted

                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                )min(1 dnss

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                r

                                                                                                                SVD Compact Representation

                                                                                                                Useful Labeling

                                                                                                                Singular Values in Increasing Order

                                                                                                                Note singular values = 0 can be omitted

                                                                                                                Let = of positive singular values

                                                                                                                )min(1 dnss

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                r

                                                                                                                SVD Compact Representation

                                                                                                                Useful Labeling

                                                                                                                Singular Values in Increasing Order

                                                                                                                Note singular values = 0 can be omitted

                                                                                                                Let = of positive singular values

                                                                                                                Then

                                                                                                                Where are truncations of

                                                                                                                trnrrrd VSUX

                                                                                                                VSU

                                                                                                                )min(1 dnss

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Full Representation

                                                                                                                =

                                                                                                                Graphics Display Assumes

                                                                                                                ndX ddU ndS nn

                                                                                                                tV

                                                                                                                nd

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Full Representation

                                                                                                                =

                                                                                                                Full Rank Basis Matrix

                                                                                                                ndX ddU ndS nn

                                                                                                                tV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Full Representation

                                                                                                                =

                                                                                                                Full Rank Basis Matrix

                                                                                                                All 0s in Bottom

                                                                                                                ndX ddU ndS nn

                                                                                                                tV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Reduced Representation

                                                                                                                =

                                                                                                                These Columns Get 0ed Out

                                                                                                                ndX ddU nnS nn

                                                                                                                tV

                                                                                                                nnd 0

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Reduced Representation

                                                                                                                =ndX ndU nnS nn

                                                                                                                tV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Reduced Representation

                                                                                                                =

                                                                                                                Also Some of These May be 0

                                                                                                                ndX ndU nnS nn

                                                                                                                tV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Compact Representation

                                                                                                                = ndX rdU

                                                                                                                rrS nrtV

                                                                                                                0

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Compact Representation

                                                                                                                =

                                                                                                                These Get 0ed Out

                                                                                                                ndX rdU

                                                                                                                rrS nrtV

                                                                                                                0

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                SVD Compact Representation

                                                                                                                = ndX rdU

                                                                                                                rrS nrtV

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition

                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                Find

                                                                                                                ddX

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition

                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                Called Eigenvalues

                                                                                                                Convenient Ordering

                                                                                                                ddX

                                                                                                                d

                                                                                                                D

                                                                                                                0

                                                                                                                01

                                                                                                                n 1

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition

                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                And an Orthonormal Matrix

                                                                                                                (ie )

                                                                                                                ddX

                                                                                                                d

                                                                                                                D

                                                                                                                0

                                                                                                                01

                                                                                                                ddB

                                                                                                                ddtt IBBBB

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition

                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                Find a Diagonal Matrix

                                                                                                                And an Orthonormal Matrix

                                                                                                                (ie )

                                                                                                                So that ie

                                                                                                                ddX

                                                                                                                d

                                                                                                                D

                                                                                                                0

                                                                                                                01

                                                                                                                ddB

                                                                                                                ddtt IBBBB

                                                                                                                DBBX tBDBX

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                (looks similar)

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                Complex (uses )

                                                                                                                VU

                                                                                                                1i

                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                VU

                                                                                                                X

                                                                                                                BVU

                                                                                                                1i

                                                                                                                • Participant Presentations
                                                                                                                • PCA to find clusters
                                                                                                                • Statistical Smoothing
                                                                                                                • Kernel Density Estimation
                                                                                                                • Scatterplot Smoothing
                                                                                                                • Statistical Smoothing (2)
                                                                                                                • SiZer Background
                                                                                                                • SiZer Background (2)
                                                                                                                • SiZer Background (3)
                                                                                                                • SiZer Background (4)
                                                                                                                • SiZer Background (5)
                                                                                                                • SiZer Overview
                                                                                                                • PCA to find clusters (2)
                                                                                                                • PCA to find clusters (3)
                                                                                                                • PCA to find clusters (4)
                                                                                                                • PCA to find clusters (5)
                                                                                                                • PCA to find clusters (6)
                                                                                                                • PCA to find clusters (7)
                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                • Frequency 2 Analysis
                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                • Yeast Cell Cycle
                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                • Reclassification of Major Genes
                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                • New Subpopulation View
                                                                                                                • New Subpopulation View (2)
                                                                                                                • Detailed Look at PCA
                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                • An Interesting Historical Note
                                                                                                                • An Interesting Historical Note (2)
                                                                                                                • Detailed Look at PCA (2)
                                                                                                                • Detailed Look at PCA (3)
                                                                                                                • Review of Linear Algebra
                                                                                                                • Review of Linear Algebra (2)
                                                                                                                • Review of Linear Algebra (3)
                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                  For matrices

                                                                                                                  Define the Matrix Product

                                                                                                                  (Inner Products of Rows With Columns )

                                                                                                                  mkk

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  nmm

                                                                                                                  n

                                                                                                                  bb

                                                                                                                  bb

                                                                                                                  B

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iniik

                                                                                                                  m

                                                                                                                  iiik

                                                                                                                  m

                                                                                                                  inii

                                                                                                                  m

                                                                                                                  iii

                                                                                                                  baba

                                                                                                                  baba

                                                                                                                  AB

                                                                                                                  1

                                                                                                                  11

                                                                                                                  11

                                                                                                                  111

                                                                                                                  A B

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                  For matrices

                                                                                                                  Define the Matrix Product

                                                                                                                  (Inner Products of Rows With Columns )

                                                                                                                  (Composition of Linear Transformations)

                                                                                                                  mkk

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  nmm

                                                                                                                  n

                                                                                                                  bb

                                                                                                                  bb

                                                                                                                  B

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iniik

                                                                                                                  m

                                                                                                                  iiik

                                                                                                                  m

                                                                                                                  inii

                                                                                                                  m

                                                                                                                  iii

                                                                                                                  baba

                                                                                                                  baba

                                                                                                                  AB

                                                                                                                  1

                                                                                                                  11

                                                                                                                  11

                                                                                                                  111

                                                                                                                  A B

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                  For matrices

                                                                                                                  Define the Matrix Product

                                                                                                                  (Inner Products of Rows With Columns )

                                                                                                                  (Composition of Linear Transformations)

                                                                                                                  Often Useful to Check Sizes

                                                                                                                  mkk

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  nmm

                                                                                                                  n

                                                                                                                  bb

                                                                                                                  bb

                                                                                                                  B

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iniik

                                                                                                                  m

                                                                                                                  iiik

                                                                                                                  m

                                                                                                                  inii

                                                                                                                  m

                                                                                                                  iii

                                                                                                                  baba

                                                                                                                  baba

                                                                                                                  AB

                                                                                                                  1

                                                                                                                  11

                                                                                                                  11

                                                                                                                  111

                                                                                                                  nmmknk

                                                                                                                  A B

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                  For matrices

                                                                                                                  Define the Matrix Product

                                                                                                                  (Inner Products of Rows With Columns )

                                                                                                                  (Composition of Linear Transformations)

                                                                                                                  Often Useful to Check Sizes

                                                                                                                  mkk

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  nmm

                                                                                                                  n

                                                                                                                  bb

                                                                                                                  bb

                                                                                                                  B

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iniik

                                                                                                                  m

                                                                                                                  iiik

                                                                                                                  m

                                                                                                                  inii

                                                                                                                  m

                                                                                                                  iii

                                                                                                                  baba

                                                                                                                  baba

                                                                                                                  AB

                                                                                                                  1

                                                                                                                  11

                                                                                                                  11

                                                                                                                  111

                                                                                                                  nmmknk

                                                                                                                  A B

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Matrix Trace

                                                                                                                  bull For a Square Matrix

                                                                                                                  bull Define

                                                                                                                  mmm

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iiiaAtr

                                                                                                                  1)(

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Matrix Trace

                                                                                                                  bull For a Square Matrix

                                                                                                                  bull Define

                                                                                                                  bull Trace Commutes with Matrix Multiplication

                                                                                                                  mmm

                                                                                                                  m

                                                                                                                  aa

                                                                                                                  aa

                                                                                                                  A

                                                                                                                  1

                                                                                                                  111

                                                                                                                  m

                                                                                                                  iiiaAtr

                                                                                                                  1)(

                                                                                                                  BAtrABtr

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                                  bull (Use Basis Above)

                                                                                                                  bull eg dim of a line is 1

                                                                                                                  bull eg dim of a plane is 2

                                                                                                                  dd dim

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                                  bull (Use Basis Above)

                                                                                                                  bull eg dim of a line is 1

                                                                                                                  bull eg dim of a plane is 2

                                                                                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                  (in Statistical Uses eg ANOVA)

                                                                                                                  dd dim

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Norm of a Vector

                                                                                                                  bull in d 21

                                                                                                                  21

                                                                                                                  1

                                                                                                                  2 xxxx td

                                                                                                                  jj

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Norm of a Vector

                                                                                                                  bull in

                                                                                                                  bull Idea length of the vector

                                                                                                                  d 2121

                                                                                                                  1

                                                                                                                  2 xxxx td

                                                                                                                  jj

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Norm of a Vector

                                                                                                                  bull in

                                                                                                                  bull Idea length of the vector

                                                                                                                  bull Note strange properties for high

                                                                                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                  d 2121

                                                                                                                  1

                                                                                                                  2 xxxx td

                                                                                                                  jj

                                                                                                                  d

                                                                                                                  d

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Norm of a Vector (cont)

                                                                                                                  bull Length Normalized Vector

                                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                                  amp is a Direction Vector)

                                                                                                                  x

                                                                                                                  x

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Norm of a Vector (cont)

                                                                                                                  bull Length Normalized Vector

                                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                                  amp is a Direction Vector)

                                                                                                                  bull Define Distance as

                                                                                                                  x

                                                                                                                  x

                                                                                                                  yxyxyxyxd t

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Inner (Dot Scalar) Product

                                                                                                                  bull for Vectors and

                                                                                                                  yxyxyx td

                                                                                                                  jjj

                                                                                                                  1

                                                                                                                  x y

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Inner (Dot Scalar) Product

                                                                                                                  bull for Vectors and

                                                                                                                  bull Related to Norm via

                                                                                                                  yxyxyx td

                                                                                                                  jjj

                                                                                                                  1

                                                                                                                  x y

                                                                                                                  21 xxx

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                  yyxx

                                                                                                                  yx

                                                                                                                  yx

                                                                                                                  yxyxangle

                                                                                                                  tt

                                                                                                                  t

                                                                                                                  11 cos

                                                                                                                  cos

                                                                                                                  x y

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                  if and only if

                                                                                                                  yyxx

                                                                                                                  yx

                                                                                                                  yx

                                                                                                                  yxyxangle

                                                                                                                  tt

                                                                                                                  t

                                                                                                                  11 cos

                                                                                                                  cos

                                                                                                                  x y

                                                                                                                  yx 0 yx

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis

                                                                                                                  bull All Orthogonal to each other

                                                                                                                  ie for

                                                                                                                  bull All have Length 1

                                                                                                                  ie for

                                                                                                                  nvv 1

                                                                                                                  1 ii vv

                                                                                                                  0 ii vv ii

                                                                                                                  ni 1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis (cont)

                                                                                                                  bull Spectral Representation

                                                                                                                  where

                                                                                                                  nvv 1

                                                                                                                  n

                                                                                                                  iii vax

                                                                                                                  1

                                                                                                                  ii vxa

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis (cont)

                                                                                                                  bull Spectral Representation

                                                                                                                  where

                                                                                                                  (Coefficient is Inner Product Cool Notation)

                                                                                                                  nvv 1

                                                                                                                  n

                                                                                                                  iii vax

                                                                                                                  1

                                                                                                                  ii vxa

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis (cont)

                                                                                                                  bull Spectral Representation

                                                                                                                  where

                                                                                                                  Check

                                                                                                                  nvv 1

                                                                                                                  n

                                                                                                                  iii vax

                                                                                                                  1

                                                                                                                  ii vxa

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iiii avvavvavx

                                                                                                                  1

                                                                                                                  1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis (cont)

                                                                                                                  bull Spectral Representation

                                                                                                                  where

                                                                                                                  Check

                                                                                                                  bull Matrix Notation where ie

                                                                                                                  For the Basis Matrix

                                                                                                                  nvv 1

                                                                                                                  n

                                                                                                                  iii vax

                                                                                                                  1

                                                                                                                  ii vxa

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iiii avvavvavx

                                                                                                                  1

                                                                                                                  1

                                                                                                                  aBx Bxa tt xBa t

                                                                                                                  nvvB 1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Orthonormal Basis (cont)

                                                                                                                  bull Spectral Representation

                                                                                                                  where

                                                                                                                  Check

                                                                                                                  bull Matrix Notation where ie

                                                                                                                  is called transform of

                                                                                                                  (eg Fourier or Wavelet)

                                                                                                                  nvv 1

                                                                                                                  n

                                                                                                                  iii vax

                                                                                                                  1

                                                                                                                  ii vxa

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iii

                                                                                                                  n

                                                                                                                  iiii avvavvavx

                                                                                                                  1

                                                                                                                  1

                                                                                                                  aBx Bxa tt xBa t

                                                                                                                  xa

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Parseval identity for

                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                  x

                                                                                                                  nvv 1

                                                                                                                  2

                                                                                                                  1

                                                                                                                  22

                                                                                                                  1

                                                                                                                  2 aavxx

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Parseval identity for

                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                  bull Pythagorean theorem

                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                  bull ANOVA - sums of squares

                                                                                                                  x

                                                                                                                  nvv 1

                                                                                                                  2

                                                                                                                  1

                                                                                                                  22

                                                                                                                  1

                                                                                                                  2 aavxx

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Parseval identity for

                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                  bull Pythagorean theorem

                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                  bull ANOVA - sums of squares

                                                                                                                  bull Transform has same length as

                                                                                                                  ie ldquorotation in rdquo

                                                                                                                  x

                                                                                                                  nvv 1

                                                                                                                  2

                                                                                                                  1

                                                                                                                  22

                                                                                                                  1

                                                                                                                  2 aavxx

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  a xd

                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                  xV

                                                                                                                  V

                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                  bull Find that Solves

                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                  xV

                                                                                                                  V

                                                                                                                  VxPV vxVv

                                                                                                                  min

                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                  bull Find that Solves

                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                  bull For Inner Product (Hilbert) Space

                                                                                                                  Exists and is Unique

                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                  xV

                                                                                                                  V

                                                                                                                  VxPV vxVv

                                                                                                                  min

                                                                                                                  xPV

                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  d VB

                                                                                                                  xBBBBxP tVV

                                                                                                                  tVVV

                                                                                                                  1

                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                  (thus projection is another linear operation)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  d VB

                                                                                                                  xBBBBxP tVV

                                                                                                                  tVVV

                                                                                                                  1

                                                                                                                  tVV

                                                                                                                  tVVV BBBBP

                                                                                                                  1

                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                  (thus projection is another linear operation)

                                                                                                                  (note same operation underlies least squares)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  d VB

                                                                                                                  xBBBBxP tVV

                                                                                                                  tVVV

                                                                                                                  1

                                                                                                                  tVV

                                                                                                                  tVVV BBBBP

                                                                                                                  1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Projection using Orthonormal Basis

                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                  nnVtV IBB

                                                                                                                  10

                                                                                                                  01

                                                                                                                  1

                                                                                                                  111

                                                                                                                  1

                                                                                                                  1

                                                                                                                  nnn

                                                                                                                  n

                                                                                                                  ntn

                                                                                                                  t

                                                                                                                  vvvv

                                                                                                                  vvvv

                                                                                                                  vv

                                                                                                                  v

                                                                                                                  v

                                                                                                                  nvv 1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Projection using Orthonormal Basis

                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                  bull So =

                                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                  (Recall Right Multrsquon)

                                                                                                                  nnVtV IBB

                                                                                                                  10

                                                                                                                  01

                                                                                                                  1

                                                                                                                  111

                                                                                                                  1

                                                                                                                  1

                                                                                                                  nnn

                                                                                                                  n

                                                                                                                  ntn

                                                                                                                  t

                                                                                                                  vvvv

                                                                                                                  vvvv

                                                                                                                  vv

                                                                                                                  v

                                                                                                                  v

                                                                                                                  xBBxP tVVV

                                                                                                                  x V

                                                                                                                  nvv 1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                  bull For Orthogonal Complement

                                                                                                                  and

                                                                                                                  V

                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                  bull For Orthogonal Complement

                                                                                                                  and

                                                                                                                  bull Parseval Inequality

                                                                                                                  V

                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                  2

                                                                                                                  1

                                                                                                                  22

                                                                                                                  1

                                                                                                                  22 aavxxxP

                                                                                                                  n

                                                                                                                  ii

                                                                                                                  n

                                                                                                                  iiV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  (Real) Unitary Matrices with

                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                  (So All of Above Applies)

                                                                                                                  ddU IUU t

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  (Real) Unitary Matrices with

                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                  (So All of Above Applies)

                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                  ddU IUU t

                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                  i ii 2

                                                                                                                  1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  (Real) Unitary Matrices with

                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                  (So All of Above Applies)

                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                  ddU IUU t

                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                  i ii 2

                                                                                                                  1

                                                                                                                  U

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                  For a Matrix

                                                                                                                  Find

                                                                                                                  ndX

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                  For a Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  with Entries

                                                                                                                  called Singular Values

                                                                                                                  ndX

                                                                                                                  ndS

                                                                                                                  )min(1 ndss

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                  For a Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  with Entries

                                                                                                                  called Singular Values

                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                  (recall )

                                                                                                                  ndX

                                                                                                                  ndS

                                                                                                                  )min(1 ndss

                                                                                                                  ddU nnV

                                                                                                                  IVVUU tt

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                  For a Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  with Entries

                                                                                                                  called Singular Values

                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                  (recall )

                                                                                                                  So That

                                                                                                                  ndX

                                                                                                                  ndS

                                                                                                                  )min(1 ndss

                                                                                                                  ddU nnV

                                                                                                                  IVVUU tt tUSVX

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                  vVSUvVSUvX tt

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                  bull First rotate

                                                                                                                  X

                                                                                                                  vVSUvVSUvX tt

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                  bull First rotate

                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                  X

                                                                                                                  vVSUvVSUvX tt

                                                                                                                  is

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                  bull First rotate

                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                  bull Third rotate again

                                                                                                                  X

                                                                                                                  vVSUvVSUvX tt

                                                                                                                  is

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                  bull First rotate

                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                  bull Third rotate again

                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                  X

                                                                                                                  vVSUvVSUvX tt

                                                                                                                  is

                                                                                                                  SVD Compact Representation

                                                                                                                  Useful Labeling

                                                                                                                  Singular Values in Increasing Order

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  )min(1 dnss

                                                                                                                  SVD Compact Representation

                                                                                                                  Useful Labeling

                                                                                                                  Singular Values in Increasing Order

                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  )min(1 dnss

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  r

                                                                                                                  SVD Compact Representation

                                                                                                                  Useful Labeling

                                                                                                                  Singular Values in Increasing Order

                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                  Let = of positive singular values

                                                                                                                  )min(1 dnss

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  r

                                                                                                                  SVD Compact Representation

                                                                                                                  Useful Labeling

                                                                                                                  Singular Values in Increasing Order

                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                  Let = of positive singular values

                                                                                                                  Then

                                                                                                                  Where are truncations of

                                                                                                                  trnrrrd VSUX

                                                                                                                  VSU

                                                                                                                  )min(1 dnss

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Full Representation

                                                                                                                  =

                                                                                                                  Graphics Display Assumes

                                                                                                                  ndX ddU ndS nn

                                                                                                                  tV

                                                                                                                  nd

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Full Representation

                                                                                                                  =

                                                                                                                  Full Rank Basis Matrix

                                                                                                                  ndX ddU ndS nn

                                                                                                                  tV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Full Representation

                                                                                                                  =

                                                                                                                  Full Rank Basis Matrix

                                                                                                                  All 0s in Bottom

                                                                                                                  ndX ddU ndS nn

                                                                                                                  tV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Reduced Representation

                                                                                                                  =

                                                                                                                  These Columns Get 0ed Out

                                                                                                                  ndX ddU nnS nn

                                                                                                                  tV

                                                                                                                  nnd 0

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Reduced Representation

                                                                                                                  =ndX ndU nnS nn

                                                                                                                  tV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Reduced Representation

                                                                                                                  =

                                                                                                                  Also Some of These May be 0

                                                                                                                  ndX ndU nnS nn

                                                                                                                  tV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Compact Representation

                                                                                                                  = ndX rdU

                                                                                                                  rrS nrtV

                                                                                                                  0

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Compact Representation

                                                                                                                  =

                                                                                                                  These Get 0ed Out

                                                                                                                  ndX rdU

                                                                                                                  rrS nrtV

                                                                                                                  0

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  SVD Compact Representation

                                                                                                                  = ndX rdU

                                                                                                                  rrS nrtV

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition

                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                  Find

                                                                                                                  ddX

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition

                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  Called Eigenvalues

                                                                                                                  Convenient Ordering

                                                                                                                  ddX

                                                                                                                  d

                                                                                                                  D

                                                                                                                  0

                                                                                                                  01

                                                                                                                  n 1

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition

                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  And an Orthonormal Matrix

                                                                                                                  (ie )

                                                                                                                  ddX

                                                                                                                  d

                                                                                                                  D

                                                                                                                  0

                                                                                                                  01

                                                                                                                  ddB

                                                                                                                  ddtt IBBBB

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition

                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                  Find a Diagonal Matrix

                                                                                                                  And an Orthonormal Matrix

                                                                                                                  (ie )

                                                                                                                  So that ie

                                                                                                                  ddX

                                                                                                                  d

                                                                                                                  D

                                                                                                                  0

                                                                                                                  01

                                                                                                                  ddB

                                                                                                                  ddtt IBBBB

                                                                                                                  DBBX tBDBX

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                  (looks similar)

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                  Complex (uses )

                                                                                                                  VU

                                                                                                                  1i

                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                  VU

                                                                                                                  X

                                                                                                                  BVU

                                                                                                                  1i

                                                                                                                  • Participant Presentations
                                                                                                                  • PCA to find clusters
                                                                                                                  • Statistical Smoothing
                                                                                                                  • Kernel Density Estimation
                                                                                                                  • Scatterplot Smoothing
                                                                                                                  • Statistical Smoothing (2)
                                                                                                                  • SiZer Background
                                                                                                                  • SiZer Background (2)
                                                                                                                  • SiZer Background (3)
                                                                                                                  • SiZer Background (4)
                                                                                                                  • SiZer Background (5)
                                                                                                                  • SiZer Overview
                                                                                                                  • PCA to find clusters (2)
                                                                                                                  • PCA to find clusters (3)
                                                                                                                  • PCA to find clusters (4)
                                                                                                                  • PCA to find clusters (5)
                                                                                                                  • PCA to find clusters (6)
                                                                                                                  • PCA to find clusters (7)
                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                  • Frequency 2 Analysis
                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                  • Yeast Cell Cycle
                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                  • Reclassification of Major Genes
                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                  • New Subpopulation View
                                                                                                                  • New Subpopulation View (2)
                                                                                                                  • Detailed Look at PCA
                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                  • An Interesting Historical Note
                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                  • Review of Linear Algebra
                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                    For matrices

                                                                                                                    Define the Matrix Product

                                                                                                                    (Inner Products of Rows With Columns )

                                                                                                                    (Composition of Linear Transformations)

                                                                                                                    mkk

                                                                                                                    m

                                                                                                                    aa

                                                                                                                    aa

                                                                                                                    A

                                                                                                                    1

                                                                                                                    111

                                                                                                                    nmm

                                                                                                                    n

                                                                                                                    bb

                                                                                                                    bb

                                                                                                                    B

                                                                                                                    1

                                                                                                                    111

                                                                                                                    m

                                                                                                                    iniik

                                                                                                                    m

                                                                                                                    iiik

                                                                                                                    m

                                                                                                                    inii

                                                                                                                    m

                                                                                                                    iii

                                                                                                                    baba

                                                                                                                    baba

                                                                                                                    AB

                                                                                                                    1

                                                                                                                    11

                                                                                                                    11

                                                                                                                    111

                                                                                                                    A B

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                    For matrices

                                                                                                                    Define the Matrix Product

                                                                                                                    (Inner Products of Rows With Columns )

                                                                                                                    (Composition of Linear Transformations)

                                                                                                                    Often Useful to Check Sizes

                                                                                                                    mkk

                                                                                                                    m

                                                                                                                    aa

                                                                                                                    aa

                                                                                                                    A

                                                                                                                    1

                                                                                                                    111

                                                                                                                    nmm

                                                                                                                    n

                                                                                                                    bb

                                                                                                                    bb

                                                                                                                    B

                                                                                                                    1

                                                                                                                    111

                                                                                                                    m

                                                                                                                    iniik

                                                                                                                    m

                                                                                                                    iiik

                                                                                                                    m

                                                                                                                    inii

                                                                                                                    m

                                                                                                                    iii

                                                                                                                    baba

                                                                                                                    baba

                                                                                                                    AB

                                                                                                                    1

                                                                                                                    11

                                                                                                                    11

                                                                                                                    111

                                                                                                                    nmmknk

                                                                                                                    A B

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                    For matrices

                                                                                                                    Define the Matrix Product

                                                                                                                    (Inner Products of Rows With Columns )

                                                                                                                    (Composition of Linear Transformations)

                                                                                                                    Often Useful to Check Sizes

                                                                                                                    mkk

                                                                                                                    m

                                                                                                                    aa

                                                                                                                    aa

                                                                                                                    A

                                                                                                                    1

                                                                                                                    111

                                                                                                                    nmm

                                                                                                                    n

                                                                                                                    bb

                                                                                                                    bb

                                                                                                                    B

                                                                                                                    1

                                                                                                                    111

                                                                                                                    m

                                                                                                                    iniik

                                                                                                                    m

                                                                                                                    iiik

                                                                                                                    m

                                                                                                                    inii

                                                                                                                    m

                                                                                                                    iii

                                                                                                                    baba

                                                                                                                    baba

                                                                                                                    AB

                                                                                                                    1

                                                                                                                    11

                                                                                                                    11

                                                                                                                    111

                                                                                                                    nmmknk

                                                                                                                    A B

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Matrix Trace

                                                                                                                    bull For a Square Matrix

                                                                                                                    bull Define

                                                                                                                    mmm

                                                                                                                    m

                                                                                                                    aa

                                                                                                                    aa

                                                                                                                    A

                                                                                                                    1

                                                                                                                    111

                                                                                                                    m

                                                                                                                    iiiaAtr

                                                                                                                    1)(

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Matrix Trace

                                                                                                                    bull For a Square Matrix

                                                                                                                    bull Define

                                                                                                                    bull Trace Commutes with Matrix Multiplication

                                                                                                                    mmm

                                                                                                                    m

                                                                                                                    aa

                                                                                                                    aa

                                                                                                                    A

                                                                                                                    1

                                                                                                                    111

                                                                                                                    m

                                                                                                                    iiiaAtr

                                                                                                                    1)(

                                                                                                                    BAtrABtr

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                                    bull (Use Basis Above)

                                                                                                                    bull eg dim of a line is 1

                                                                                                                    bull eg dim of a plane is 2

                                                                                                                    dd dim

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                    bull Number of Elements in a Basis (Unique)

                                                                                                                    bull (Use Basis Above)

                                                                                                                    bull eg dim of a line is 1

                                                                                                                    bull eg dim of a plane is 2

                                                                                                                    bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                    (in Statistical Uses eg ANOVA)

                                                                                                                    dd dim

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Norm of a Vector

                                                                                                                    bull in d 21

                                                                                                                    21

                                                                                                                    1

                                                                                                                    2 xxxx td

                                                                                                                    jj

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Norm of a Vector

                                                                                                                    bull in

                                                                                                                    bull Idea length of the vector

                                                                                                                    d 2121

                                                                                                                    1

                                                                                                                    2 xxxx td

                                                                                                                    jj

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Norm of a Vector

                                                                                                                    bull in

                                                                                                                    bull Idea length of the vector

                                                                                                                    bull Note strange properties for high

                                                                                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                    d 2121

                                                                                                                    1

                                                                                                                    2 xxxx td

                                                                                                                    jj

                                                                                                                    d

                                                                                                                    d

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Norm of a Vector (cont)

                                                                                                                    bull Length Normalized Vector

                                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                                    amp is a Direction Vector)

                                                                                                                    x

                                                                                                                    x

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Norm of a Vector (cont)

                                                                                                                    bull Length Normalized Vector

                                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                                    amp is a Direction Vector)

                                                                                                                    bull Define Distance as

                                                                                                                    x

                                                                                                                    x

                                                                                                                    yxyxyxyxd t

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Inner (Dot Scalar) Product

                                                                                                                    bull for Vectors and

                                                                                                                    yxyxyx td

                                                                                                                    jjj

                                                                                                                    1

                                                                                                                    x y

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Inner (Dot Scalar) Product

                                                                                                                    bull for Vectors and

                                                                                                                    bull Related to Norm via

                                                                                                                    yxyxyx td

                                                                                                                    jjj

                                                                                                                    1

                                                                                                                    x y

                                                                                                                    21 xxx

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                                    yyxx

                                                                                                                    yx

                                                                                                                    yx

                                                                                                                    yxyxangle

                                                                                                                    tt

                                                                                                                    t

                                                                                                                    11 cos

                                                                                                                    cos

                                                                                                                    x y

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                    if and only if

                                                                                                                    yyxx

                                                                                                                    yx

                                                                                                                    yx

                                                                                                                    yxyxangle

                                                                                                                    tt

                                                                                                                    t

                                                                                                                    11 cos

                                                                                                                    cos

                                                                                                                    x y

                                                                                                                    yx 0 yx

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis

                                                                                                                    bull All Orthogonal to each other

                                                                                                                    ie for

                                                                                                                    bull All have Length 1

                                                                                                                    ie for

                                                                                                                    nvv 1

                                                                                                                    1 ii vv

                                                                                                                    0 ii vv ii

                                                                                                                    ni 1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis (cont)

                                                                                                                    bull Spectral Representation

                                                                                                                    where

                                                                                                                    nvv 1

                                                                                                                    n

                                                                                                                    iii vax

                                                                                                                    1

                                                                                                                    ii vxa

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis (cont)

                                                                                                                    bull Spectral Representation

                                                                                                                    where

                                                                                                                    (Coefficient is Inner Product Cool Notation)

                                                                                                                    nvv 1

                                                                                                                    n

                                                                                                                    iii vax

                                                                                                                    1

                                                                                                                    ii vxa

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis (cont)

                                                                                                                    bull Spectral Representation

                                                                                                                    where

                                                                                                                    Check

                                                                                                                    nvv 1

                                                                                                                    n

                                                                                                                    iii vax

                                                                                                                    1

                                                                                                                    ii vxa

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iiii avvavvavx

                                                                                                                    1

                                                                                                                    1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis (cont)

                                                                                                                    bull Spectral Representation

                                                                                                                    where

                                                                                                                    Check

                                                                                                                    bull Matrix Notation where ie

                                                                                                                    For the Basis Matrix

                                                                                                                    nvv 1

                                                                                                                    n

                                                                                                                    iii vax

                                                                                                                    1

                                                                                                                    ii vxa

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iiii avvavvavx

                                                                                                                    1

                                                                                                                    1

                                                                                                                    aBx Bxa tt xBa t

                                                                                                                    nvvB 1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Orthonormal Basis (cont)

                                                                                                                    bull Spectral Representation

                                                                                                                    where

                                                                                                                    Check

                                                                                                                    bull Matrix Notation where ie

                                                                                                                    is called transform of

                                                                                                                    (eg Fourier or Wavelet)

                                                                                                                    nvv 1

                                                                                                                    n

                                                                                                                    iii vax

                                                                                                                    1

                                                                                                                    ii vxa

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iii

                                                                                                                    n

                                                                                                                    iiii avvavvavx

                                                                                                                    1

                                                                                                                    1

                                                                                                                    aBx Bxa tt xBa t

                                                                                                                    xa

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Parseval identity for

                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                    x

                                                                                                                    nvv 1

                                                                                                                    2

                                                                                                                    1

                                                                                                                    22

                                                                                                                    1

                                                                                                                    2 aavxx

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Parseval identity for

                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                    bull Pythagorean theorem

                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                    bull ANOVA - sums of squares

                                                                                                                    x

                                                                                                                    nvv 1

                                                                                                                    2

                                                                                                                    1

                                                                                                                    22

                                                                                                                    1

                                                                                                                    2 aavxx

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Parseval identity for

                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                    bull Pythagorean theorem

                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                    bull ANOVA - sums of squares

                                                                                                                    bull Transform has same length as

                                                                                                                    ie ldquorotation in rdquo

                                                                                                                    x

                                                                                                                    nvv 1

                                                                                                                    2

                                                                                                                    1

                                                                                                                    22

                                                                                                                    1

                                                                                                                    2 aavxx

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    a xd

                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                    xV

                                                                                                                    V

                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                    bull Find that Solves

                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                    xV

                                                                                                                    V

                                                                                                                    VxPV vxVv

                                                                                                                    min

                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                    bull Find that Solves

                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                    bull For Inner Product (Hilbert) Space

                                                                                                                    Exists and is Unique

                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                    xV

                                                                                                                    V

                                                                                                                    VxPV vxVv

                                                                                                                    min

                                                                                                                    xPV

                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    d VB

                                                                                                                    xBBBBxP tVV

                                                                                                                    tVVV

                                                                                                                    1

                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                    (thus projection is another linear operation)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    d VB

                                                                                                                    xBBBBxP tVV

                                                                                                                    tVVV

                                                                                                                    1

                                                                                                                    tVV

                                                                                                                    tVVV BBBBP

                                                                                                                    1

                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                    (thus projection is another linear operation)

                                                                                                                    (note same operation underlies least squares)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    d VB

                                                                                                                    xBBBBxP tVV

                                                                                                                    tVVV

                                                                                                                    1

                                                                                                                    tVV

                                                                                                                    tVVV BBBBP

                                                                                                                    1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Projection using Orthonormal Basis

                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                    nnVtV IBB

                                                                                                                    10

                                                                                                                    01

                                                                                                                    1

                                                                                                                    111

                                                                                                                    1

                                                                                                                    1

                                                                                                                    nnn

                                                                                                                    n

                                                                                                                    ntn

                                                                                                                    t

                                                                                                                    vvvv

                                                                                                                    vvvv

                                                                                                                    vv

                                                                                                                    v

                                                                                                                    v

                                                                                                                    nvv 1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Projection using Orthonormal Basis

                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                    bull So =

                                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                    (Recall Right Multrsquon)

                                                                                                                    nnVtV IBB

                                                                                                                    10

                                                                                                                    01

                                                                                                                    1

                                                                                                                    111

                                                                                                                    1

                                                                                                                    1

                                                                                                                    nnn

                                                                                                                    n

                                                                                                                    ntn

                                                                                                                    t

                                                                                                                    vvvv

                                                                                                                    vvvv

                                                                                                                    vv

                                                                                                                    v

                                                                                                                    v

                                                                                                                    xBBxP tVVV

                                                                                                                    x V

                                                                                                                    nvv 1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                    bull For Orthogonal Complement

                                                                                                                    and

                                                                                                                    V

                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                    bull For Orthogonal Complement

                                                                                                                    and

                                                                                                                    bull Parseval Inequality

                                                                                                                    V

                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                    2

                                                                                                                    1

                                                                                                                    22

                                                                                                                    1

                                                                                                                    22 aavxxxP

                                                                                                                    n

                                                                                                                    ii

                                                                                                                    n

                                                                                                                    iiV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    (Real) Unitary Matrices with

                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                    (So All of Above Applies)

                                                                                                                    ddU IUU t

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    (Real) Unitary Matrices with

                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                    (So All of Above Applies)

                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                    ddU IUU t

                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                    i ii 2

                                                                                                                    1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    (Real) Unitary Matrices with

                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                    (So All of Above Applies)

                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                    ddU IUU t

                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                    i ii 2

                                                                                                                    1

                                                                                                                    U

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                    For a Matrix

                                                                                                                    Find

                                                                                                                    ndX

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                    For a Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    with Entries

                                                                                                                    called Singular Values

                                                                                                                    ndX

                                                                                                                    ndS

                                                                                                                    )min(1 ndss

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                    For a Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    with Entries

                                                                                                                    called Singular Values

                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                    (recall )

                                                                                                                    ndX

                                                                                                                    ndS

                                                                                                                    )min(1 ndss

                                                                                                                    ddU nnV

                                                                                                                    IVVUU tt

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                    For a Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    with Entries

                                                                                                                    called Singular Values

                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                    (recall )

                                                                                                                    So That

                                                                                                                    ndX

                                                                                                                    ndS

                                                                                                                    )min(1 ndss

                                                                                                                    ddU nnV

                                                                                                                    IVVUU tt tUSVX

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                    vVSUvVSUvX tt

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                    bull First rotate

                                                                                                                    X

                                                                                                                    vVSUvVSUvX tt

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                    bull First rotate

                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                    X

                                                                                                                    vVSUvVSUvX tt

                                                                                                                    is

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                    bull First rotate

                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                    bull Third rotate again

                                                                                                                    X

                                                                                                                    vVSUvVSUvX tt

                                                                                                                    is

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                    bull First rotate

                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                    bull Third rotate again

                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                    X

                                                                                                                    vVSUvVSUvX tt

                                                                                                                    is

                                                                                                                    SVD Compact Representation

                                                                                                                    Useful Labeling

                                                                                                                    Singular Values in Increasing Order

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    )min(1 dnss

                                                                                                                    SVD Compact Representation

                                                                                                                    Useful Labeling

                                                                                                                    Singular Values in Increasing Order

                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    )min(1 dnss

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    r

                                                                                                                    SVD Compact Representation

                                                                                                                    Useful Labeling

                                                                                                                    Singular Values in Increasing Order

                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                    Let = of positive singular values

                                                                                                                    )min(1 dnss

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    r

                                                                                                                    SVD Compact Representation

                                                                                                                    Useful Labeling

                                                                                                                    Singular Values in Increasing Order

                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                    Let = of positive singular values

                                                                                                                    Then

                                                                                                                    Where are truncations of

                                                                                                                    trnrrrd VSUX

                                                                                                                    VSU

                                                                                                                    )min(1 dnss

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Full Representation

                                                                                                                    =

                                                                                                                    Graphics Display Assumes

                                                                                                                    ndX ddU ndS nn

                                                                                                                    tV

                                                                                                                    nd

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Full Representation

                                                                                                                    =

                                                                                                                    Full Rank Basis Matrix

                                                                                                                    ndX ddU ndS nn

                                                                                                                    tV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Full Representation

                                                                                                                    =

                                                                                                                    Full Rank Basis Matrix

                                                                                                                    All 0s in Bottom

                                                                                                                    ndX ddU ndS nn

                                                                                                                    tV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Reduced Representation

                                                                                                                    =

                                                                                                                    These Columns Get 0ed Out

                                                                                                                    ndX ddU nnS nn

                                                                                                                    tV

                                                                                                                    nnd 0

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Reduced Representation

                                                                                                                    =ndX ndU nnS nn

                                                                                                                    tV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Reduced Representation

                                                                                                                    =

                                                                                                                    Also Some of These May be 0

                                                                                                                    ndX ndU nnS nn

                                                                                                                    tV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Compact Representation

                                                                                                                    = ndX rdU

                                                                                                                    rrS nrtV

                                                                                                                    0

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Compact Representation

                                                                                                                    =

                                                                                                                    These Get 0ed Out

                                                                                                                    ndX rdU

                                                                                                                    rrS nrtV

                                                                                                                    0

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    SVD Compact Representation

                                                                                                                    = ndX rdU

                                                                                                                    rrS nrtV

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition

                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                    Find

                                                                                                                    ddX

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition

                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    Called Eigenvalues

                                                                                                                    Convenient Ordering

                                                                                                                    ddX

                                                                                                                    d

                                                                                                                    D

                                                                                                                    0

                                                                                                                    01

                                                                                                                    n 1

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition

                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    And an Orthonormal Matrix

                                                                                                                    (ie )

                                                                                                                    ddX

                                                                                                                    d

                                                                                                                    D

                                                                                                                    0

                                                                                                                    01

                                                                                                                    ddB

                                                                                                                    ddtt IBBBB

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition

                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                    Find a Diagonal Matrix

                                                                                                                    And an Orthonormal Matrix

                                                                                                                    (ie )

                                                                                                                    So that ie

                                                                                                                    ddX

                                                                                                                    d

                                                                                                                    D

                                                                                                                    0

                                                                                                                    01

                                                                                                                    ddB

                                                                                                                    ddtt IBBBB

                                                                                                                    DBBX tBDBX

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                    (looks similar)

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                    Complex (uses )

                                                                                                                    VU

                                                                                                                    1i

                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                    VU

                                                                                                                    X

                                                                                                                    BVU

                                                                                                                    1i

                                                                                                                    • Participant Presentations
                                                                                                                    • PCA to find clusters
                                                                                                                    • Statistical Smoothing
                                                                                                                    • Kernel Density Estimation
                                                                                                                    • Scatterplot Smoothing
                                                                                                                    • Statistical Smoothing (2)
                                                                                                                    • SiZer Background
                                                                                                                    • SiZer Background (2)
                                                                                                                    • SiZer Background (3)
                                                                                                                    • SiZer Background (4)
                                                                                                                    • SiZer Background (5)
                                                                                                                    • SiZer Overview
                                                                                                                    • PCA to find clusters (2)
                                                                                                                    • PCA to find clusters (3)
                                                                                                                    • PCA to find clusters (4)
                                                                                                                    • PCA to find clusters (5)
                                                                                                                    • PCA to find clusters (6)
                                                                                                                    • PCA to find clusters (7)
                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                    • Frequency 2 Analysis
                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                    • Yeast Cell Cycle
                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                    • Reclassification of Major Genes
                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                    • New Subpopulation View
                                                                                                                    • New Subpopulation View (2)
                                                                                                                    • Detailed Look at PCA
                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                    • An Interesting Historical Note
                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                    • Review of Linear Algebra
                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                      For matrices

                                                                                                                      Define the Matrix Product

                                                                                                                      (Inner Products of Rows With Columns )

                                                                                                                      (Composition of Linear Transformations)

                                                                                                                      Often Useful to Check Sizes

                                                                                                                      mkk

                                                                                                                      m

                                                                                                                      aa

                                                                                                                      aa

                                                                                                                      A

                                                                                                                      1

                                                                                                                      111

                                                                                                                      nmm

                                                                                                                      n

                                                                                                                      bb

                                                                                                                      bb

                                                                                                                      B

                                                                                                                      1

                                                                                                                      111

                                                                                                                      m

                                                                                                                      iniik

                                                                                                                      m

                                                                                                                      iiik

                                                                                                                      m

                                                                                                                      inii

                                                                                                                      m

                                                                                                                      iii

                                                                                                                      baba

                                                                                                                      baba

                                                                                                                      AB

                                                                                                                      1

                                                                                                                      11

                                                                                                                      11

                                                                                                                      111

                                                                                                                      nmmknk

                                                                                                                      A B

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                      For matrices

                                                                                                                      Define the Matrix Product

                                                                                                                      (Inner Products of Rows With Columns )

                                                                                                                      (Composition of Linear Transformations)

                                                                                                                      Often Useful to Check Sizes

                                                                                                                      mkk

                                                                                                                      m

                                                                                                                      aa

                                                                                                                      aa

                                                                                                                      A

                                                                                                                      1

                                                                                                                      111

                                                                                                                      nmm

                                                                                                                      n

                                                                                                                      bb

                                                                                                                      bb

                                                                                                                      B

                                                                                                                      1

                                                                                                                      111

                                                                                                                      m

                                                                                                                      iniik

                                                                                                                      m

                                                                                                                      iiik

                                                                                                                      m

                                                                                                                      inii

                                                                                                                      m

                                                                                                                      iii

                                                                                                                      baba

                                                                                                                      baba

                                                                                                                      AB

                                                                                                                      1

                                                                                                                      11

                                                                                                                      11

                                                                                                                      111

                                                                                                                      nmmknk

                                                                                                                      A B

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Matrix Trace

                                                                                                                      bull For a Square Matrix

                                                                                                                      bull Define

                                                                                                                      mmm

                                                                                                                      m

                                                                                                                      aa

                                                                                                                      aa

                                                                                                                      A

                                                                                                                      1

                                                                                                                      111

                                                                                                                      m

                                                                                                                      iiiaAtr

                                                                                                                      1)(

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Matrix Trace

                                                                                                                      bull For a Square Matrix

                                                                                                                      bull Define

                                                                                                                      bull Trace Commutes with Matrix Multiplication

                                                                                                                      mmm

                                                                                                                      m

                                                                                                                      aa

                                                                                                                      aa

                                                                                                                      A

                                                                                                                      1

                                                                                                                      111

                                                                                                                      m

                                                                                                                      iiiaAtr

                                                                                                                      1)(

                                                                                                                      BAtrABtr

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                                      bull (Use Basis Above)

                                                                                                                      bull eg dim of a line is 1

                                                                                                                      bull eg dim of a plane is 2

                                                                                                                      dd dim

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                      bull Number of Elements in a Basis (Unique)

                                                                                                                      bull (Use Basis Above)

                                                                                                                      bull eg dim of a line is 1

                                                                                                                      bull eg dim of a plane is 2

                                                                                                                      bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                      (in Statistical Uses eg ANOVA)

                                                                                                                      dd dim

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Norm of a Vector

                                                                                                                      bull in d 21

                                                                                                                      21

                                                                                                                      1

                                                                                                                      2 xxxx td

                                                                                                                      jj

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Norm of a Vector

                                                                                                                      bull in

                                                                                                                      bull Idea length of the vector

                                                                                                                      d 2121

                                                                                                                      1

                                                                                                                      2 xxxx td

                                                                                                                      jj

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Norm of a Vector

                                                                                                                      bull in

                                                                                                                      bull Idea length of the vector

                                                                                                                      bull Note strange properties for high

                                                                                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                      d 2121

                                                                                                                      1

                                                                                                                      2 xxxx td

                                                                                                                      jj

                                                                                                                      d

                                                                                                                      d

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Norm of a Vector (cont)

                                                                                                                      bull Length Normalized Vector

                                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                                      amp is a Direction Vector)

                                                                                                                      x

                                                                                                                      x

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Norm of a Vector (cont)

                                                                                                                      bull Length Normalized Vector

                                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                                      amp is a Direction Vector)

                                                                                                                      bull Define Distance as

                                                                                                                      x

                                                                                                                      x

                                                                                                                      yxyxyxyxd t

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Inner (Dot Scalar) Product

                                                                                                                      bull for Vectors and

                                                                                                                      yxyxyx td

                                                                                                                      jjj

                                                                                                                      1

                                                                                                                      x y

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Inner (Dot Scalar) Product

                                                                                                                      bull for Vectors and

                                                                                                                      bull Related to Norm via

                                                                                                                      yxyxyx td

                                                                                                                      jjj

                                                                                                                      1

                                                                                                                      x y

                                                                                                                      21 xxx

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                                      yyxx

                                                                                                                      yx

                                                                                                                      yx

                                                                                                                      yxyxangle

                                                                                                                      tt

                                                                                                                      t

                                                                                                                      11 cos

                                                                                                                      cos

                                                                                                                      x y

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                      if and only if

                                                                                                                      yyxx

                                                                                                                      yx

                                                                                                                      yx

                                                                                                                      yxyxangle

                                                                                                                      tt

                                                                                                                      t

                                                                                                                      11 cos

                                                                                                                      cos

                                                                                                                      x y

                                                                                                                      yx 0 yx

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis

                                                                                                                      bull All Orthogonal to each other

                                                                                                                      ie for

                                                                                                                      bull All have Length 1

                                                                                                                      ie for

                                                                                                                      nvv 1

                                                                                                                      1 ii vv

                                                                                                                      0 ii vv ii

                                                                                                                      ni 1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis (cont)

                                                                                                                      bull Spectral Representation

                                                                                                                      where

                                                                                                                      nvv 1

                                                                                                                      n

                                                                                                                      iii vax

                                                                                                                      1

                                                                                                                      ii vxa

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis (cont)

                                                                                                                      bull Spectral Representation

                                                                                                                      where

                                                                                                                      (Coefficient is Inner Product Cool Notation)

                                                                                                                      nvv 1

                                                                                                                      n

                                                                                                                      iii vax

                                                                                                                      1

                                                                                                                      ii vxa

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis (cont)

                                                                                                                      bull Spectral Representation

                                                                                                                      where

                                                                                                                      Check

                                                                                                                      nvv 1

                                                                                                                      n

                                                                                                                      iii vax

                                                                                                                      1

                                                                                                                      ii vxa

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iiii avvavvavx

                                                                                                                      1

                                                                                                                      1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis (cont)

                                                                                                                      bull Spectral Representation

                                                                                                                      where

                                                                                                                      Check

                                                                                                                      bull Matrix Notation where ie

                                                                                                                      For the Basis Matrix

                                                                                                                      nvv 1

                                                                                                                      n

                                                                                                                      iii vax

                                                                                                                      1

                                                                                                                      ii vxa

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iiii avvavvavx

                                                                                                                      1

                                                                                                                      1

                                                                                                                      aBx Bxa tt xBa t

                                                                                                                      nvvB 1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Orthonormal Basis (cont)

                                                                                                                      bull Spectral Representation

                                                                                                                      where

                                                                                                                      Check

                                                                                                                      bull Matrix Notation where ie

                                                                                                                      is called transform of

                                                                                                                      (eg Fourier or Wavelet)

                                                                                                                      nvv 1

                                                                                                                      n

                                                                                                                      iii vax

                                                                                                                      1

                                                                                                                      ii vxa

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iii

                                                                                                                      n

                                                                                                                      iiii avvavvavx

                                                                                                                      1

                                                                                                                      1

                                                                                                                      aBx Bxa tt xBa t

                                                                                                                      xa

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Parseval identity for

                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                      x

                                                                                                                      nvv 1

                                                                                                                      2

                                                                                                                      1

                                                                                                                      22

                                                                                                                      1

                                                                                                                      2 aavxx

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Parseval identity for

                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                      bull Pythagorean theorem

                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                      bull ANOVA - sums of squares

                                                                                                                      x

                                                                                                                      nvv 1

                                                                                                                      2

                                                                                                                      1

                                                                                                                      22

                                                                                                                      1

                                                                                                                      2 aavxx

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Parseval identity for

                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                      bull Pythagorean theorem

                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                      bull ANOVA - sums of squares

                                                                                                                      bull Transform has same length as

                                                                                                                      ie ldquorotation in rdquo

                                                                                                                      x

                                                                                                                      nvv 1

                                                                                                                      2

                                                                                                                      1

                                                                                                                      22

                                                                                                                      1

                                                                                                                      2 aavxx

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      a xd

                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                      xV

                                                                                                                      V

                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                      bull Find that Solves

                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                      xV

                                                                                                                      V

                                                                                                                      VxPV vxVv

                                                                                                                      min

                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                      bull Find that Solves

                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                      bull For Inner Product (Hilbert) Space

                                                                                                                      Exists and is Unique

                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                      xV

                                                                                                                      V

                                                                                                                      VxPV vxVv

                                                                                                                      min

                                                                                                                      xPV

                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      d VB

                                                                                                                      xBBBBxP tVV

                                                                                                                      tVVV

                                                                                                                      1

                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                      (thus projection is another linear operation)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      d VB

                                                                                                                      xBBBBxP tVV

                                                                                                                      tVVV

                                                                                                                      1

                                                                                                                      tVV

                                                                                                                      tVVV BBBBP

                                                                                                                      1

                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                      (thus projection is another linear operation)

                                                                                                                      (note same operation underlies least squares)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      d VB

                                                                                                                      xBBBBxP tVV

                                                                                                                      tVVV

                                                                                                                      1

                                                                                                                      tVV

                                                                                                                      tVVV BBBBP

                                                                                                                      1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Projection using Orthonormal Basis

                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                      nnVtV IBB

                                                                                                                      10

                                                                                                                      01

                                                                                                                      1

                                                                                                                      111

                                                                                                                      1

                                                                                                                      1

                                                                                                                      nnn

                                                                                                                      n

                                                                                                                      ntn

                                                                                                                      t

                                                                                                                      vvvv

                                                                                                                      vvvv

                                                                                                                      vv

                                                                                                                      v

                                                                                                                      v

                                                                                                                      nvv 1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Projection using Orthonormal Basis

                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                      bull So =

                                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                      (Recall Right Multrsquon)

                                                                                                                      nnVtV IBB

                                                                                                                      10

                                                                                                                      01

                                                                                                                      1

                                                                                                                      111

                                                                                                                      1

                                                                                                                      1

                                                                                                                      nnn

                                                                                                                      n

                                                                                                                      ntn

                                                                                                                      t

                                                                                                                      vvvv

                                                                                                                      vvvv

                                                                                                                      vv

                                                                                                                      v

                                                                                                                      v

                                                                                                                      xBBxP tVVV

                                                                                                                      x V

                                                                                                                      nvv 1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                      bull For Orthogonal Complement

                                                                                                                      and

                                                                                                                      V

                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                      bull For Orthogonal Complement

                                                                                                                      and

                                                                                                                      bull Parseval Inequality

                                                                                                                      V

                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                      2

                                                                                                                      1

                                                                                                                      22

                                                                                                                      1

                                                                                                                      22 aavxxxP

                                                                                                                      n

                                                                                                                      ii

                                                                                                                      n

                                                                                                                      iiV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      (Real) Unitary Matrices with

                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                      (So All of Above Applies)

                                                                                                                      ddU IUU t

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      (Real) Unitary Matrices with

                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                      (So All of Above Applies)

                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                      ddU IUU t

                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                      i ii 2

                                                                                                                      1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      (Real) Unitary Matrices with

                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                      (So All of Above Applies)

                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                      ddU IUU t

                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                      i ii 2

                                                                                                                      1

                                                                                                                      U

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                      For a Matrix

                                                                                                                      Find

                                                                                                                      ndX

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                      For a Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      with Entries

                                                                                                                      called Singular Values

                                                                                                                      ndX

                                                                                                                      ndS

                                                                                                                      )min(1 ndss

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                      For a Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      with Entries

                                                                                                                      called Singular Values

                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                      (recall )

                                                                                                                      ndX

                                                                                                                      ndS

                                                                                                                      )min(1 ndss

                                                                                                                      ddU nnV

                                                                                                                      IVVUU tt

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                      For a Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      with Entries

                                                                                                                      called Singular Values

                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                      (recall )

                                                                                                                      So That

                                                                                                                      ndX

                                                                                                                      ndS

                                                                                                                      )min(1 ndss

                                                                                                                      ddU nnV

                                                                                                                      IVVUU tt tUSVX

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                      vVSUvVSUvX tt

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                      bull First rotate

                                                                                                                      X

                                                                                                                      vVSUvVSUvX tt

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                      bull First rotate

                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                      X

                                                                                                                      vVSUvVSUvX tt

                                                                                                                      is

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                      bull First rotate

                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                      bull Third rotate again

                                                                                                                      X

                                                                                                                      vVSUvVSUvX tt

                                                                                                                      is

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                      bull First rotate

                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                      bull Third rotate again

                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                      X

                                                                                                                      vVSUvVSUvX tt

                                                                                                                      is

                                                                                                                      SVD Compact Representation

                                                                                                                      Useful Labeling

                                                                                                                      Singular Values in Increasing Order

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      )min(1 dnss

                                                                                                                      SVD Compact Representation

                                                                                                                      Useful Labeling

                                                                                                                      Singular Values in Increasing Order

                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      )min(1 dnss

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      r

                                                                                                                      SVD Compact Representation

                                                                                                                      Useful Labeling

                                                                                                                      Singular Values in Increasing Order

                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                      Let = of positive singular values

                                                                                                                      )min(1 dnss

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      r

                                                                                                                      SVD Compact Representation

                                                                                                                      Useful Labeling

                                                                                                                      Singular Values in Increasing Order

                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                      Let = of positive singular values

                                                                                                                      Then

                                                                                                                      Where are truncations of

                                                                                                                      trnrrrd VSUX

                                                                                                                      VSU

                                                                                                                      )min(1 dnss

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Full Representation

                                                                                                                      =

                                                                                                                      Graphics Display Assumes

                                                                                                                      ndX ddU ndS nn

                                                                                                                      tV

                                                                                                                      nd

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Full Representation

                                                                                                                      =

                                                                                                                      Full Rank Basis Matrix

                                                                                                                      ndX ddU ndS nn

                                                                                                                      tV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Full Representation

                                                                                                                      =

                                                                                                                      Full Rank Basis Matrix

                                                                                                                      All 0s in Bottom

                                                                                                                      ndX ddU ndS nn

                                                                                                                      tV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Reduced Representation

                                                                                                                      =

                                                                                                                      These Columns Get 0ed Out

                                                                                                                      ndX ddU nnS nn

                                                                                                                      tV

                                                                                                                      nnd 0

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Reduced Representation

                                                                                                                      =ndX ndU nnS nn

                                                                                                                      tV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Reduced Representation

                                                                                                                      =

                                                                                                                      Also Some of These May be 0

                                                                                                                      ndX ndU nnS nn

                                                                                                                      tV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Compact Representation

                                                                                                                      = ndX rdU

                                                                                                                      rrS nrtV

                                                                                                                      0

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Compact Representation

                                                                                                                      =

                                                                                                                      These Get 0ed Out

                                                                                                                      ndX rdU

                                                                                                                      rrS nrtV

                                                                                                                      0

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      SVD Compact Representation

                                                                                                                      = ndX rdU

                                                                                                                      rrS nrtV

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition

                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                      Find

                                                                                                                      ddX

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition

                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      Called Eigenvalues

                                                                                                                      Convenient Ordering

                                                                                                                      ddX

                                                                                                                      d

                                                                                                                      D

                                                                                                                      0

                                                                                                                      01

                                                                                                                      n 1

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition

                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      And an Orthonormal Matrix

                                                                                                                      (ie )

                                                                                                                      ddX

                                                                                                                      d

                                                                                                                      D

                                                                                                                      0

                                                                                                                      01

                                                                                                                      ddB

                                                                                                                      ddtt IBBBB

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition

                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                      Find a Diagonal Matrix

                                                                                                                      And an Orthonormal Matrix

                                                                                                                      (ie )

                                                                                                                      So that ie

                                                                                                                      ddX

                                                                                                                      d

                                                                                                                      D

                                                                                                                      0

                                                                                                                      01

                                                                                                                      ddB

                                                                                                                      ddtt IBBBB

                                                                                                                      DBBX tBDBX

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                      (looks similar)

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                      Complex (uses )

                                                                                                                      VU

                                                                                                                      1i

                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                      VU

                                                                                                                      X

                                                                                                                      BVU

                                                                                                                      1i

                                                                                                                      • Participant Presentations
                                                                                                                      • PCA to find clusters
                                                                                                                      • Statistical Smoothing
                                                                                                                      • Kernel Density Estimation
                                                                                                                      • Scatterplot Smoothing
                                                                                                                      • Statistical Smoothing (2)
                                                                                                                      • SiZer Background
                                                                                                                      • SiZer Background (2)
                                                                                                                      • SiZer Background (3)
                                                                                                                      • SiZer Background (4)
                                                                                                                      • SiZer Background (5)
                                                                                                                      • SiZer Overview
                                                                                                                      • PCA to find clusters (2)
                                                                                                                      • PCA to find clusters (3)
                                                                                                                      • PCA to find clusters (4)
                                                                                                                      • PCA to find clusters (5)
                                                                                                                      • PCA to find clusters (6)
                                                                                                                      • PCA to find clusters (7)
                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                      • Frequency 2 Analysis
                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                      • Yeast Cell Cycle
                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                      • Reclassification of Major Genes
                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                      • New Subpopulation View
                                                                                                                      • New Subpopulation View (2)
                                                                                                                      • Detailed Look at PCA
                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                      • An Interesting Historical Note
                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                      • Review of Linear Algebra
                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Aside on Matrix Multiplication (linear transformatrsquon)

                                                                                                                        For matrices

                                                                                                                        Define the Matrix Product

                                                                                                                        (Inner Products of Rows With Columns )

                                                                                                                        (Composition of Linear Transformations)

                                                                                                                        Often Useful to Check Sizes

                                                                                                                        mkk

                                                                                                                        m

                                                                                                                        aa

                                                                                                                        aa

                                                                                                                        A

                                                                                                                        1

                                                                                                                        111

                                                                                                                        nmm

                                                                                                                        n

                                                                                                                        bb

                                                                                                                        bb

                                                                                                                        B

                                                                                                                        1

                                                                                                                        111

                                                                                                                        m

                                                                                                                        iniik

                                                                                                                        m

                                                                                                                        iiik

                                                                                                                        m

                                                                                                                        inii

                                                                                                                        m

                                                                                                                        iii

                                                                                                                        baba

                                                                                                                        baba

                                                                                                                        AB

                                                                                                                        1

                                                                                                                        11

                                                                                                                        11

                                                                                                                        111

                                                                                                                        nmmknk

                                                                                                                        A B

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Matrix Trace

                                                                                                                        bull For a Square Matrix

                                                                                                                        bull Define

                                                                                                                        mmm

                                                                                                                        m

                                                                                                                        aa

                                                                                                                        aa

                                                                                                                        A

                                                                                                                        1

                                                                                                                        111

                                                                                                                        m

                                                                                                                        iiiaAtr

                                                                                                                        1)(

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Matrix Trace

                                                                                                                        bull For a Square Matrix

                                                                                                                        bull Define

                                                                                                                        bull Trace Commutes with Matrix Multiplication

                                                                                                                        mmm

                                                                                                                        m

                                                                                                                        aa

                                                                                                                        aa

                                                                                                                        A

                                                                                                                        1

                                                                                                                        111

                                                                                                                        m

                                                                                                                        iiiaAtr

                                                                                                                        1)(

                                                                                                                        BAtrABtr

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                                        bull (Use Basis Above)

                                                                                                                        bull eg dim of a line is 1

                                                                                                                        bull eg dim of a plane is 2

                                                                                                                        dd dim

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                        bull Number of Elements in a Basis (Unique)

                                                                                                                        bull (Use Basis Above)

                                                                                                                        bull eg dim of a line is 1

                                                                                                                        bull eg dim of a plane is 2

                                                                                                                        bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                        (in Statistical Uses eg ANOVA)

                                                                                                                        dd dim

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Norm of a Vector

                                                                                                                        bull in d 21

                                                                                                                        21

                                                                                                                        1

                                                                                                                        2 xxxx td

                                                                                                                        jj

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Norm of a Vector

                                                                                                                        bull in

                                                                                                                        bull Idea length of the vector

                                                                                                                        d 2121

                                                                                                                        1

                                                                                                                        2 xxxx td

                                                                                                                        jj

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Norm of a Vector

                                                                                                                        bull in

                                                                                                                        bull Idea length of the vector

                                                                                                                        bull Note strange properties for high

                                                                                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                        d 2121

                                                                                                                        1

                                                                                                                        2 xxxx td

                                                                                                                        jj

                                                                                                                        d

                                                                                                                        d

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Norm of a Vector (cont)

                                                                                                                        bull Length Normalized Vector

                                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                                        amp is a Direction Vector)

                                                                                                                        x

                                                                                                                        x

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Norm of a Vector (cont)

                                                                                                                        bull Length Normalized Vector

                                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                                        amp is a Direction Vector)

                                                                                                                        bull Define Distance as

                                                                                                                        x

                                                                                                                        x

                                                                                                                        yxyxyxyxd t

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Inner (Dot Scalar) Product

                                                                                                                        bull for Vectors and

                                                                                                                        yxyxyx td

                                                                                                                        jjj

                                                                                                                        1

                                                                                                                        x y

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Inner (Dot Scalar) Product

                                                                                                                        bull for Vectors and

                                                                                                                        bull Related to Norm via

                                                                                                                        yxyxyx td

                                                                                                                        jjj

                                                                                                                        1

                                                                                                                        x y

                                                                                                                        21 xxx

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                                        yyxx

                                                                                                                        yx

                                                                                                                        yx

                                                                                                                        yxyxangle

                                                                                                                        tt

                                                                                                                        t

                                                                                                                        11 cos

                                                                                                                        cos

                                                                                                                        x y

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                        if and only if

                                                                                                                        yyxx

                                                                                                                        yx

                                                                                                                        yx

                                                                                                                        yxyxangle

                                                                                                                        tt

                                                                                                                        t

                                                                                                                        11 cos

                                                                                                                        cos

                                                                                                                        x y

                                                                                                                        yx 0 yx

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis

                                                                                                                        bull All Orthogonal to each other

                                                                                                                        ie for

                                                                                                                        bull All have Length 1

                                                                                                                        ie for

                                                                                                                        nvv 1

                                                                                                                        1 ii vv

                                                                                                                        0 ii vv ii

                                                                                                                        ni 1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis (cont)

                                                                                                                        bull Spectral Representation

                                                                                                                        where

                                                                                                                        nvv 1

                                                                                                                        n

                                                                                                                        iii vax

                                                                                                                        1

                                                                                                                        ii vxa

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis (cont)

                                                                                                                        bull Spectral Representation

                                                                                                                        where

                                                                                                                        (Coefficient is Inner Product Cool Notation)

                                                                                                                        nvv 1

                                                                                                                        n

                                                                                                                        iii vax

                                                                                                                        1

                                                                                                                        ii vxa

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis (cont)

                                                                                                                        bull Spectral Representation

                                                                                                                        where

                                                                                                                        Check

                                                                                                                        nvv 1

                                                                                                                        n

                                                                                                                        iii vax

                                                                                                                        1

                                                                                                                        ii vxa

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iiii avvavvavx

                                                                                                                        1

                                                                                                                        1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis (cont)

                                                                                                                        bull Spectral Representation

                                                                                                                        where

                                                                                                                        Check

                                                                                                                        bull Matrix Notation where ie

                                                                                                                        For the Basis Matrix

                                                                                                                        nvv 1

                                                                                                                        n

                                                                                                                        iii vax

                                                                                                                        1

                                                                                                                        ii vxa

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iiii avvavvavx

                                                                                                                        1

                                                                                                                        1

                                                                                                                        aBx Bxa tt xBa t

                                                                                                                        nvvB 1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Orthonormal Basis (cont)

                                                                                                                        bull Spectral Representation

                                                                                                                        where

                                                                                                                        Check

                                                                                                                        bull Matrix Notation where ie

                                                                                                                        is called transform of

                                                                                                                        (eg Fourier or Wavelet)

                                                                                                                        nvv 1

                                                                                                                        n

                                                                                                                        iii vax

                                                                                                                        1

                                                                                                                        ii vxa

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iii

                                                                                                                        n

                                                                                                                        iiii avvavvavx

                                                                                                                        1

                                                                                                                        1

                                                                                                                        aBx Bxa tt xBa t

                                                                                                                        xa

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Parseval identity for

                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                        x

                                                                                                                        nvv 1

                                                                                                                        2

                                                                                                                        1

                                                                                                                        22

                                                                                                                        1

                                                                                                                        2 aavxx

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Parseval identity for

                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                        bull Pythagorean theorem

                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                        bull ANOVA - sums of squares

                                                                                                                        x

                                                                                                                        nvv 1

                                                                                                                        2

                                                                                                                        1

                                                                                                                        22

                                                                                                                        1

                                                                                                                        2 aavxx

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Parseval identity for

                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                        bull Pythagorean theorem

                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                        bull ANOVA - sums of squares

                                                                                                                        bull Transform has same length as

                                                                                                                        ie ldquorotation in rdquo

                                                                                                                        x

                                                                                                                        nvv 1

                                                                                                                        2

                                                                                                                        1

                                                                                                                        22

                                                                                                                        1

                                                                                                                        2 aavxx

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        a xd

                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                        xV

                                                                                                                        V

                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                        bull Find that Solves

                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                        xV

                                                                                                                        V

                                                                                                                        VxPV vxVv

                                                                                                                        min

                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                        bull Find that Solves

                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                        bull For Inner Product (Hilbert) Space

                                                                                                                        Exists and is Unique

                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                        xV

                                                                                                                        V

                                                                                                                        VxPV vxVv

                                                                                                                        min

                                                                                                                        xPV

                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        d VB

                                                                                                                        xBBBBxP tVV

                                                                                                                        tVVV

                                                                                                                        1

                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                        (thus projection is another linear operation)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        d VB

                                                                                                                        xBBBBxP tVV

                                                                                                                        tVVV

                                                                                                                        1

                                                                                                                        tVV

                                                                                                                        tVVV BBBBP

                                                                                                                        1

                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                        (thus projection is another linear operation)

                                                                                                                        (note same operation underlies least squares)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        d VB

                                                                                                                        xBBBBxP tVV

                                                                                                                        tVVV

                                                                                                                        1

                                                                                                                        tVV

                                                                                                                        tVVV BBBBP

                                                                                                                        1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Projection using Orthonormal Basis

                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                        nnVtV IBB

                                                                                                                        10

                                                                                                                        01

                                                                                                                        1

                                                                                                                        111

                                                                                                                        1

                                                                                                                        1

                                                                                                                        nnn

                                                                                                                        n

                                                                                                                        ntn

                                                                                                                        t

                                                                                                                        vvvv

                                                                                                                        vvvv

                                                                                                                        vv

                                                                                                                        v

                                                                                                                        v

                                                                                                                        nvv 1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Projection using Orthonormal Basis

                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                        bull So =

                                                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                        (Recall Right Multrsquon)

                                                                                                                        nnVtV IBB

                                                                                                                        10

                                                                                                                        01

                                                                                                                        1

                                                                                                                        111

                                                                                                                        1

                                                                                                                        1

                                                                                                                        nnn

                                                                                                                        n

                                                                                                                        ntn

                                                                                                                        t

                                                                                                                        vvvv

                                                                                                                        vvvv

                                                                                                                        vv

                                                                                                                        v

                                                                                                                        v

                                                                                                                        xBBxP tVVV

                                                                                                                        x V

                                                                                                                        nvv 1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                        bull For Orthogonal Complement

                                                                                                                        and

                                                                                                                        V

                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                        bull For Orthogonal Complement

                                                                                                                        and

                                                                                                                        bull Parseval Inequality

                                                                                                                        V

                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                        2

                                                                                                                        1

                                                                                                                        22

                                                                                                                        1

                                                                                                                        22 aavxxxP

                                                                                                                        n

                                                                                                                        ii

                                                                                                                        n

                                                                                                                        iiV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        (Real) Unitary Matrices with

                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                        (So All of Above Applies)

                                                                                                                        ddU IUU t

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        (Real) Unitary Matrices with

                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                        (So All of Above Applies)

                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                        ddU IUU t

                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                        i ii 2

                                                                                                                        1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        (Real) Unitary Matrices with

                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                        (So All of Above Applies)

                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                        ddU IUU t

                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                        i ii 2

                                                                                                                        1

                                                                                                                        U

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                        For a Matrix

                                                                                                                        Find

                                                                                                                        ndX

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                        For a Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        with Entries

                                                                                                                        called Singular Values

                                                                                                                        ndX

                                                                                                                        ndS

                                                                                                                        )min(1 ndss

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                        For a Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        with Entries

                                                                                                                        called Singular Values

                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                        (recall )

                                                                                                                        ndX

                                                                                                                        ndS

                                                                                                                        )min(1 ndss

                                                                                                                        ddU nnV

                                                                                                                        IVVUU tt

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                        For a Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        with Entries

                                                                                                                        called Singular Values

                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                        (recall )

                                                                                                                        So That

                                                                                                                        ndX

                                                                                                                        ndS

                                                                                                                        )min(1 ndss

                                                                                                                        ddU nnV

                                                                                                                        IVVUU tt tUSVX

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                        vVSUvVSUvX tt

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                        bull First rotate

                                                                                                                        X

                                                                                                                        vVSUvVSUvX tt

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                        bull First rotate

                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                        X

                                                                                                                        vVSUvVSUvX tt

                                                                                                                        is

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                        bull First rotate

                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                        bull Third rotate again

                                                                                                                        X

                                                                                                                        vVSUvVSUvX tt

                                                                                                                        is

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                        bull First rotate

                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                        bull Third rotate again

                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                        X

                                                                                                                        vVSUvVSUvX tt

                                                                                                                        is

                                                                                                                        SVD Compact Representation

                                                                                                                        Useful Labeling

                                                                                                                        Singular Values in Increasing Order

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        )min(1 dnss

                                                                                                                        SVD Compact Representation

                                                                                                                        Useful Labeling

                                                                                                                        Singular Values in Increasing Order

                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        )min(1 dnss

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        r

                                                                                                                        SVD Compact Representation

                                                                                                                        Useful Labeling

                                                                                                                        Singular Values in Increasing Order

                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                        Let = of positive singular values

                                                                                                                        )min(1 dnss

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        r

                                                                                                                        SVD Compact Representation

                                                                                                                        Useful Labeling

                                                                                                                        Singular Values in Increasing Order

                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                        Let = of positive singular values

                                                                                                                        Then

                                                                                                                        Where are truncations of

                                                                                                                        trnrrrd VSUX

                                                                                                                        VSU

                                                                                                                        )min(1 dnss

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Full Representation

                                                                                                                        =

                                                                                                                        Graphics Display Assumes

                                                                                                                        ndX ddU ndS nn

                                                                                                                        tV

                                                                                                                        nd

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Full Representation

                                                                                                                        =

                                                                                                                        Full Rank Basis Matrix

                                                                                                                        ndX ddU ndS nn

                                                                                                                        tV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Full Representation

                                                                                                                        =

                                                                                                                        Full Rank Basis Matrix

                                                                                                                        All 0s in Bottom

                                                                                                                        ndX ddU ndS nn

                                                                                                                        tV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Reduced Representation

                                                                                                                        =

                                                                                                                        These Columns Get 0ed Out

                                                                                                                        ndX ddU nnS nn

                                                                                                                        tV

                                                                                                                        nnd 0

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Reduced Representation

                                                                                                                        =ndX ndU nnS nn

                                                                                                                        tV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Reduced Representation

                                                                                                                        =

                                                                                                                        Also Some of These May be 0

                                                                                                                        ndX ndU nnS nn

                                                                                                                        tV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Compact Representation

                                                                                                                        = ndX rdU

                                                                                                                        rrS nrtV

                                                                                                                        0

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Compact Representation

                                                                                                                        =

                                                                                                                        These Get 0ed Out

                                                                                                                        ndX rdU

                                                                                                                        rrS nrtV

                                                                                                                        0

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        SVD Compact Representation

                                                                                                                        = ndX rdU

                                                                                                                        rrS nrtV

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition

                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                        Find

                                                                                                                        ddX

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition

                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        Called Eigenvalues

                                                                                                                        Convenient Ordering

                                                                                                                        ddX

                                                                                                                        d

                                                                                                                        D

                                                                                                                        0

                                                                                                                        01

                                                                                                                        n 1

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition

                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        And an Orthonormal Matrix

                                                                                                                        (ie )

                                                                                                                        ddX

                                                                                                                        d

                                                                                                                        D

                                                                                                                        0

                                                                                                                        01

                                                                                                                        ddB

                                                                                                                        ddtt IBBBB

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition

                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                        Find a Diagonal Matrix

                                                                                                                        And an Orthonormal Matrix

                                                                                                                        (ie )

                                                                                                                        So that ie

                                                                                                                        ddX

                                                                                                                        d

                                                                                                                        D

                                                                                                                        0

                                                                                                                        01

                                                                                                                        ddB

                                                                                                                        ddtt IBBBB

                                                                                                                        DBBX tBDBX

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                        (looks similar)

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                        Complex (uses )

                                                                                                                        VU

                                                                                                                        1i

                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                        VU

                                                                                                                        X

                                                                                                                        BVU

                                                                                                                        1i

                                                                                                                        • Participant Presentations
                                                                                                                        • PCA to find clusters
                                                                                                                        • Statistical Smoothing
                                                                                                                        • Kernel Density Estimation
                                                                                                                        • Scatterplot Smoothing
                                                                                                                        • Statistical Smoothing (2)
                                                                                                                        • SiZer Background
                                                                                                                        • SiZer Background (2)
                                                                                                                        • SiZer Background (3)
                                                                                                                        • SiZer Background (4)
                                                                                                                        • SiZer Background (5)
                                                                                                                        • SiZer Overview
                                                                                                                        • PCA to find clusters (2)
                                                                                                                        • PCA to find clusters (3)
                                                                                                                        • PCA to find clusters (4)
                                                                                                                        • PCA to find clusters (5)
                                                                                                                        • PCA to find clusters (6)
                                                                                                                        • PCA to find clusters (7)
                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                        • Frequency 2 Analysis
                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                        • Yeast Cell Cycle
                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                        • Reclassification of Major Genes
                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                        • New Subpopulation View
                                                                                                                        • New Subpopulation View (2)
                                                                                                                        • Detailed Look at PCA
                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                        • An Interesting Historical Note
                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                        • Review of Linear Algebra
                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Matrix Trace

                                                                                                                          bull For a Square Matrix

                                                                                                                          bull Define

                                                                                                                          mmm

                                                                                                                          m

                                                                                                                          aa

                                                                                                                          aa

                                                                                                                          A

                                                                                                                          1

                                                                                                                          111

                                                                                                                          m

                                                                                                                          iiiaAtr

                                                                                                                          1)(

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Matrix Trace

                                                                                                                          bull For a Square Matrix

                                                                                                                          bull Define

                                                                                                                          bull Trace Commutes with Matrix Multiplication

                                                                                                                          mmm

                                                                                                                          m

                                                                                                                          aa

                                                                                                                          aa

                                                                                                                          A

                                                                                                                          1

                                                                                                                          111

                                                                                                                          m

                                                                                                                          iiiaAtr

                                                                                                                          1)(

                                                                                                                          BAtrABtr

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                                          bull (Use Basis Above)

                                                                                                                          bull eg dim of a line is 1

                                                                                                                          bull eg dim of a plane is 2

                                                                                                                          dd dim

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                          bull Number of Elements in a Basis (Unique)

                                                                                                                          bull (Use Basis Above)

                                                                                                                          bull eg dim of a line is 1

                                                                                                                          bull eg dim of a plane is 2

                                                                                                                          bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                          (in Statistical Uses eg ANOVA)

                                                                                                                          dd dim

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Norm of a Vector

                                                                                                                          bull in d 21

                                                                                                                          21

                                                                                                                          1

                                                                                                                          2 xxxx td

                                                                                                                          jj

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Norm of a Vector

                                                                                                                          bull in

                                                                                                                          bull Idea length of the vector

                                                                                                                          d 2121

                                                                                                                          1

                                                                                                                          2 xxxx td

                                                                                                                          jj

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Norm of a Vector

                                                                                                                          bull in

                                                                                                                          bull Idea length of the vector

                                                                                                                          bull Note strange properties for high

                                                                                                                          eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                          d 2121

                                                                                                                          1

                                                                                                                          2 xxxx td

                                                                                                                          jj

                                                                                                                          d

                                                                                                                          d

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Norm of a Vector (cont)

                                                                                                                          bull Length Normalized Vector

                                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                                          amp is a Direction Vector)

                                                                                                                          x

                                                                                                                          x

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Norm of a Vector (cont)

                                                                                                                          bull Length Normalized Vector

                                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                                          amp is a Direction Vector)

                                                                                                                          bull Define Distance as

                                                                                                                          x

                                                                                                                          x

                                                                                                                          yxyxyxyxd t

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Inner (Dot Scalar) Product

                                                                                                                          bull for Vectors and

                                                                                                                          yxyxyx td

                                                                                                                          jjj

                                                                                                                          1

                                                                                                                          x y

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Inner (Dot Scalar) Product

                                                                                                                          bull for Vectors and

                                                                                                                          bull Related to Norm via

                                                                                                                          yxyxyx td

                                                                                                                          jjj

                                                                                                                          1

                                                                                                                          x y

                                                                                                                          21 xxx

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                                          yyxx

                                                                                                                          yx

                                                                                                                          yx

                                                                                                                          yxyxangle

                                                                                                                          tt

                                                                                                                          t

                                                                                                                          11 cos

                                                                                                                          cos

                                                                                                                          x y

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                          if and only if

                                                                                                                          yyxx

                                                                                                                          yx

                                                                                                                          yx

                                                                                                                          yxyxangle

                                                                                                                          tt

                                                                                                                          t

                                                                                                                          11 cos

                                                                                                                          cos

                                                                                                                          x y

                                                                                                                          yx 0 yx

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis

                                                                                                                          bull All Orthogonal to each other

                                                                                                                          ie for

                                                                                                                          bull All have Length 1

                                                                                                                          ie for

                                                                                                                          nvv 1

                                                                                                                          1 ii vv

                                                                                                                          0 ii vv ii

                                                                                                                          ni 1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis (cont)

                                                                                                                          bull Spectral Representation

                                                                                                                          where

                                                                                                                          nvv 1

                                                                                                                          n

                                                                                                                          iii vax

                                                                                                                          1

                                                                                                                          ii vxa

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis (cont)

                                                                                                                          bull Spectral Representation

                                                                                                                          where

                                                                                                                          (Coefficient is Inner Product Cool Notation)

                                                                                                                          nvv 1

                                                                                                                          n

                                                                                                                          iii vax

                                                                                                                          1

                                                                                                                          ii vxa

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis (cont)

                                                                                                                          bull Spectral Representation

                                                                                                                          where

                                                                                                                          Check

                                                                                                                          nvv 1

                                                                                                                          n

                                                                                                                          iii vax

                                                                                                                          1

                                                                                                                          ii vxa

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iiii avvavvavx

                                                                                                                          1

                                                                                                                          1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis (cont)

                                                                                                                          bull Spectral Representation

                                                                                                                          where

                                                                                                                          Check

                                                                                                                          bull Matrix Notation where ie

                                                                                                                          For the Basis Matrix

                                                                                                                          nvv 1

                                                                                                                          n

                                                                                                                          iii vax

                                                                                                                          1

                                                                                                                          ii vxa

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iiii avvavvavx

                                                                                                                          1

                                                                                                                          1

                                                                                                                          aBx Bxa tt xBa t

                                                                                                                          nvvB 1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Orthonormal Basis (cont)

                                                                                                                          bull Spectral Representation

                                                                                                                          where

                                                                                                                          Check

                                                                                                                          bull Matrix Notation where ie

                                                                                                                          is called transform of

                                                                                                                          (eg Fourier or Wavelet)

                                                                                                                          nvv 1

                                                                                                                          n

                                                                                                                          iii vax

                                                                                                                          1

                                                                                                                          ii vxa

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iii

                                                                                                                          n

                                                                                                                          iiii avvavvavx

                                                                                                                          1

                                                                                                                          1

                                                                                                                          aBx Bxa tt xBa t

                                                                                                                          xa

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Parseval identity for

                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                          x

                                                                                                                          nvv 1

                                                                                                                          2

                                                                                                                          1

                                                                                                                          22

                                                                                                                          1

                                                                                                                          2 aavxx

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Parseval identity for

                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                          bull Pythagorean theorem

                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                          bull ANOVA - sums of squares

                                                                                                                          x

                                                                                                                          nvv 1

                                                                                                                          2

                                                                                                                          1

                                                                                                                          22

                                                                                                                          1

                                                                                                                          2 aavxx

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Parseval identity for

                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                          bull Pythagorean theorem

                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                          bull ANOVA - sums of squares

                                                                                                                          bull Transform has same length as

                                                                                                                          ie ldquorotation in rdquo

                                                                                                                          x

                                                                                                                          nvv 1

                                                                                                                          2

                                                                                                                          1

                                                                                                                          22

                                                                                                                          1

                                                                                                                          2 aavxx

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          a xd

                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                          xV

                                                                                                                          V

                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                          bull Find that Solves

                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                          xV

                                                                                                                          V

                                                                                                                          VxPV vxVv

                                                                                                                          min

                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                          bull Find that Solves

                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                          bull For Inner Product (Hilbert) Space

                                                                                                                          Exists and is Unique

                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                          xV

                                                                                                                          V

                                                                                                                          VxPV vxVv

                                                                                                                          min

                                                                                                                          xPV

                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          d VB

                                                                                                                          xBBBBxP tVV

                                                                                                                          tVVV

                                                                                                                          1

                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                          (thus projection is another linear operation)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          d VB

                                                                                                                          xBBBBxP tVV

                                                                                                                          tVVV

                                                                                                                          1

                                                                                                                          tVV

                                                                                                                          tVVV BBBBP

                                                                                                                          1

                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                          (thus projection is another linear operation)

                                                                                                                          (note same operation underlies least squares)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          d VB

                                                                                                                          xBBBBxP tVV

                                                                                                                          tVVV

                                                                                                                          1

                                                                                                                          tVV

                                                                                                                          tVVV BBBBP

                                                                                                                          1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Projection using Orthonormal Basis

                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                          nnVtV IBB

                                                                                                                          10

                                                                                                                          01

                                                                                                                          1

                                                                                                                          111

                                                                                                                          1

                                                                                                                          1

                                                                                                                          nnn

                                                                                                                          n

                                                                                                                          ntn

                                                                                                                          t

                                                                                                                          vvvv

                                                                                                                          vvvv

                                                                                                                          vv

                                                                                                                          v

                                                                                                                          v

                                                                                                                          nvv 1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Projection using Orthonormal Basis

                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                          bull So =

                                                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                          (Recall Right Multrsquon)

                                                                                                                          nnVtV IBB

                                                                                                                          10

                                                                                                                          01

                                                                                                                          1

                                                                                                                          111

                                                                                                                          1

                                                                                                                          1

                                                                                                                          nnn

                                                                                                                          n

                                                                                                                          ntn

                                                                                                                          t

                                                                                                                          vvvv

                                                                                                                          vvvv

                                                                                                                          vv

                                                                                                                          v

                                                                                                                          v

                                                                                                                          xBBxP tVVV

                                                                                                                          x V

                                                                                                                          nvv 1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                          bull For Orthogonal Complement

                                                                                                                          and

                                                                                                                          V

                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                          bull For Orthogonal Complement

                                                                                                                          and

                                                                                                                          bull Parseval Inequality

                                                                                                                          V

                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                          2

                                                                                                                          1

                                                                                                                          22

                                                                                                                          1

                                                                                                                          22 aavxxxP

                                                                                                                          n

                                                                                                                          ii

                                                                                                                          n

                                                                                                                          iiV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          (Real) Unitary Matrices with

                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                          (So All of Above Applies)

                                                                                                                          ddU IUU t

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          (Real) Unitary Matrices with

                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                          (So All of Above Applies)

                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                          ddU IUU t

                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                          i ii 2

                                                                                                                          1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          (Real) Unitary Matrices with

                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                          (So All of Above Applies)

                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                          ddU IUU t

                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                          i ii 2

                                                                                                                          1

                                                                                                                          U

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                          For a Matrix

                                                                                                                          Find

                                                                                                                          ndX

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                          For a Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          with Entries

                                                                                                                          called Singular Values

                                                                                                                          ndX

                                                                                                                          ndS

                                                                                                                          )min(1 ndss

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                          For a Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          with Entries

                                                                                                                          called Singular Values

                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                          (recall )

                                                                                                                          ndX

                                                                                                                          ndS

                                                                                                                          )min(1 ndss

                                                                                                                          ddU nnV

                                                                                                                          IVVUU tt

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                          For a Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          with Entries

                                                                                                                          called Singular Values

                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                          (recall )

                                                                                                                          So That

                                                                                                                          ndX

                                                                                                                          ndS

                                                                                                                          )min(1 ndss

                                                                                                                          ddU nnV

                                                                                                                          IVVUU tt tUSVX

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                          vVSUvVSUvX tt

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                          bull First rotate

                                                                                                                          X

                                                                                                                          vVSUvVSUvX tt

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                          bull First rotate

                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                          X

                                                                                                                          vVSUvVSUvX tt

                                                                                                                          is

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                          bull First rotate

                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                          bull Third rotate again

                                                                                                                          X

                                                                                                                          vVSUvVSUvX tt

                                                                                                                          is

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                          bull First rotate

                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                          bull Third rotate again

                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                          X

                                                                                                                          vVSUvVSUvX tt

                                                                                                                          is

                                                                                                                          SVD Compact Representation

                                                                                                                          Useful Labeling

                                                                                                                          Singular Values in Increasing Order

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          )min(1 dnss

                                                                                                                          SVD Compact Representation

                                                                                                                          Useful Labeling

                                                                                                                          Singular Values in Increasing Order

                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          )min(1 dnss

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          r

                                                                                                                          SVD Compact Representation

                                                                                                                          Useful Labeling

                                                                                                                          Singular Values in Increasing Order

                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                          Let = of positive singular values

                                                                                                                          )min(1 dnss

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          r

                                                                                                                          SVD Compact Representation

                                                                                                                          Useful Labeling

                                                                                                                          Singular Values in Increasing Order

                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                          Let = of positive singular values

                                                                                                                          Then

                                                                                                                          Where are truncations of

                                                                                                                          trnrrrd VSUX

                                                                                                                          VSU

                                                                                                                          )min(1 dnss

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Full Representation

                                                                                                                          =

                                                                                                                          Graphics Display Assumes

                                                                                                                          ndX ddU ndS nn

                                                                                                                          tV

                                                                                                                          nd

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Full Representation

                                                                                                                          =

                                                                                                                          Full Rank Basis Matrix

                                                                                                                          ndX ddU ndS nn

                                                                                                                          tV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Full Representation

                                                                                                                          =

                                                                                                                          Full Rank Basis Matrix

                                                                                                                          All 0s in Bottom

                                                                                                                          ndX ddU ndS nn

                                                                                                                          tV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Reduced Representation

                                                                                                                          =

                                                                                                                          These Columns Get 0ed Out

                                                                                                                          ndX ddU nnS nn

                                                                                                                          tV

                                                                                                                          nnd 0

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Reduced Representation

                                                                                                                          =ndX ndU nnS nn

                                                                                                                          tV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Reduced Representation

                                                                                                                          =

                                                                                                                          Also Some of These May be 0

                                                                                                                          ndX ndU nnS nn

                                                                                                                          tV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Compact Representation

                                                                                                                          = ndX rdU

                                                                                                                          rrS nrtV

                                                                                                                          0

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Compact Representation

                                                                                                                          =

                                                                                                                          These Get 0ed Out

                                                                                                                          ndX rdU

                                                                                                                          rrS nrtV

                                                                                                                          0

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          SVD Compact Representation

                                                                                                                          = ndX rdU

                                                                                                                          rrS nrtV

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition

                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                          Find

                                                                                                                          ddX

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition

                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          Called Eigenvalues

                                                                                                                          Convenient Ordering

                                                                                                                          ddX

                                                                                                                          d

                                                                                                                          D

                                                                                                                          0

                                                                                                                          01

                                                                                                                          n 1

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition

                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          And an Orthonormal Matrix

                                                                                                                          (ie )

                                                                                                                          ddX

                                                                                                                          d

                                                                                                                          D

                                                                                                                          0

                                                                                                                          01

                                                                                                                          ddB

                                                                                                                          ddtt IBBBB

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition

                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                          Find a Diagonal Matrix

                                                                                                                          And an Orthonormal Matrix

                                                                                                                          (ie )

                                                                                                                          So that ie

                                                                                                                          ddX

                                                                                                                          d

                                                                                                                          D

                                                                                                                          0

                                                                                                                          01

                                                                                                                          ddB

                                                                                                                          ddtt IBBBB

                                                                                                                          DBBX tBDBX

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                          (looks similar)

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                          Complex (uses )

                                                                                                                          VU

                                                                                                                          1i

                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                          VU

                                                                                                                          X

                                                                                                                          BVU

                                                                                                                          1i

                                                                                                                          • Participant Presentations
                                                                                                                          • PCA to find clusters
                                                                                                                          • Statistical Smoothing
                                                                                                                          • Kernel Density Estimation
                                                                                                                          • Scatterplot Smoothing
                                                                                                                          • Statistical Smoothing (2)
                                                                                                                          • SiZer Background
                                                                                                                          • SiZer Background (2)
                                                                                                                          • SiZer Background (3)
                                                                                                                          • SiZer Background (4)
                                                                                                                          • SiZer Background (5)
                                                                                                                          • SiZer Overview
                                                                                                                          • PCA to find clusters (2)
                                                                                                                          • PCA to find clusters (3)
                                                                                                                          • PCA to find clusters (4)
                                                                                                                          • PCA to find clusters (5)
                                                                                                                          • PCA to find clusters (6)
                                                                                                                          • PCA to find clusters (7)
                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                          • Frequency 2 Analysis
                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                          • Yeast Cell Cycle
                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                          • Reclassification of Major Genes
                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                          • New Subpopulation View
                                                                                                                          • New Subpopulation View (2)
                                                                                                                          • Detailed Look at PCA
                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                          • An Interesting Historical Note
                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                          • Review of Linear Algebra
                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Matrix Trace

                                                                                                                            bull For a Square Matrix

                                                                                                                            bull Define

                                                                                                                            bull Trace Commutes with Matrix Multiplication

                                                                                                                            mmm

                                                                                                                            m

                                                                                                                            aa

                                                                                                                            aa

                                                                                                                            A

                                                                                                                            1

                                                                                                                            111

                                                                                                                            m

                                                                                                                            iiiaAtr

                                                                                                                            1)(

                                                                                                                            BAtrABtr

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                                            bull (Use Basis Above)

                                                                                                                            bull eg dim of a line is 1

                                                                                                                            bull eg dim of a plane is 2

                                                                                                                            dd dim

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                            bull Number of Elements in a Basis (Unique)

                                                                                                                            bull (Use Basis Above)

                                                                                                                            bull eg dim of a line is 1

                                                                                                                            bull eg dim of a plane is 2

                                                                                                                            bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                            (in Statistical Uses eg ANOVA)

                                                                                                                            dd dim

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Norm of a Vector

                                                                                                                            bull in d 21

                                                                                                                            21

                                                                                                                            1

                                                                                                                            2 xxxx td

                                                                                                                            jj

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Norm of a Vector

                                                                                                                            bull in

                                                                                                                            bull Idea length of the vector

                                                                                                                            d 2121

                                                                                                                            1

                                                                                                                            2 xxxx td

                                                                                                                            jj

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Norm of a Vector

                                                                                                                            bull in

                                                                                                                            bull Idea length of the vector

                                                                                                                            bull Note strange properties for high

                                                                                                                            eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                            d 2121

                                                                                                                            1

                                                                                                                            2 xxxx td

                                                                                                                            jj

                                                                                                                            d

                                                                                                                            d

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Norm of a Vector (cont)

                                                                                                                            bull Length Normalized Vector

                                                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                                                            amp is a Direction Vector)

                                                                                                                            x

                                                                                                                            x

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Norm of a Vector (cont)

                                                                                                                            bull Length Normalized Vector

                                                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                                                            amp is a Direction Vector)

                                                                                                                            bull Define Distance as

                                                                                                                            x

                                                                                                                            x

                                                                                                                            yxyxyxyxd t

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Inner (Dot Scalar) Product

                                                                                                                            bull for Vectors and

                                                                                                                            yxyxyx td

                                                                                                                            jjj

                                                                                                                            1

                                                                                                                            x y

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Inner (Dot Scalar) Product

                                                                                                                            bull for Vectors and

                                                                                                                            bull Related to Norm via

                                                                                                                            yxyxyx td

                                                                                                                            jjj

                                                                                                                            1

                                                                                                                            x y

                                                                                                                            21 xxx

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                                            yyxx

                                                                                                                            yx

                                                                                                                            yx

                                                                                                                            yxyxangle

                                                                                                                            tt

                                                                                                                            t

                                                                                                                            11 cos

                                                                                                                            cos

                                                                                                                            x y

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                            if and only if

                                                                                                                            yyxx

                                                                                                                            yx

                                                                                                                            yx

                                                                                                                            yxyxangle

                                                                                                                            tt

                                                                                                                            t

                                                                                                                            11 cos

                                                                                                                            cos

                                                                                                                            x y

                                                                                                                            yx 0 yx

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis

                                                                                                                            bull All Orthogonal to each other

                                                                                                                            ie for

                                                                                                                            bull All have Length 1

                                                                                                                            ie for

                                                                                                                            nvv 1

                                                                                                                            1 ii vv

                                                                                                                            0 ii vv ii

                                                                                                                            ni 1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis (cont)

                                                                                                                            bull Spectral Representation

                                                                                                                            where

                                                                                                                            nvv 1

                                                                                                                            n

                                                                                                                            iii vax

                                                                                                                            1

                                                                                                                            ii vxa

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis (cont)

                                                                                                                            bull Spectral Representation

                                                                                                                            where

                                                                                                                            (Coefficient is Inner Product Cool Notation)

                                                                                                                            nvv 1

                                                                                                                            n

                                                                                                                            iii vax

                                                                                                                            1

                                                                                                                            ii vxa

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis (cont)

                                                                                                                            bull Spectral Representation

                                                                                                                            where

                                                                                                                            Check

                                                                                                                            nvv 1

                                                                                                                            n

                                                                                                                            iii vax

                                                                                                                            1

                                                                                                                            ii vxa

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iiii avvavvavx

                                                                                                                            1

                                                                                                                            1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis (cont)

                                                                                                                            bull Spectral Representation

                                                                                                                            where

                                                                                                                            Check

                                                                                                                            bull Matrix Notation where ie

                                                                                                                            For the Basis Matrix

                                                                                                                            nvv 1

                                                                                                                            n

                                                                                                                            iii vax

                                                                                                                            1

                                                                                                                            ii vxa

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iiii avvavvavx

                                                                                                                            1

                                                                                                                            1

                                                                                                                            aBx Bxa tt xBa t

                                                                                                                            nvvB 1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Orthonormal Basis (cont)

                                                                                                                            bull Spectral Representation

                                                                                                                            where

                                                                                                                            Check

                                                                                                                            bull Matrix Notation where ie

                                                                                                                            is called transform of

                                                                                                                            (eg Fourier or Wavelet)

                                                                                                                            nvv 1

                                                                                                                            n

                                                                                                                            iii vax

                                                                                                                            1

                                                                                                                            ii vxa

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iii

                                                                                                                            n

                                                                                                                            iiii avvavvavx

                                                                                                                            1

                                                                                                                            1

                                                                                                                            aBx Bxa tt xBa t

                                                                                                                            xa

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Parseval identity for

                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                            x

                                                                                                                            nvv 1

                                                                                                                            2

                                                                                                                            1

                                                                                                                            22

                                                                                                                            1

                                                                                                                            2 aavxx

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Parseval identity for

                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                            bull Pythagorean theorem

                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                            bull ANOVA - sums of squares

                                                                                                                            x

                                                                                                                            nvv 1

                                                                                                                            2

                                                                                                                            1

                                                                                                                            22

                                                                                                                            1

                                                                                                                            2 aavxx

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Parseval identity for

                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                            bull Pythagorean theorem

                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                            bull ANOVA - sums of squares

                                                                                                                            bull Transform has same length as

                                                                                                                            ie ldquorotation in rdquo

                                                                                                                            x

                                                                                                                            nvv 1

                                                                                                                            2

                                                                                                                            1

                                                                                                                            22

                                                                                                                            1

                                                                                                                            2 aavxx

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            a xd

                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                            xV

                                                                                                                            V

                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                            bull Find that Solves

                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                            xV

                                                                                                                            V

                                                                                                                            VxPV vxVv

                                                                                                                            min

                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                            bull Find that Solves

                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                            bull For Inner Product (Hilbert) Space

                                                                                                                            Exists and is Unique

                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                            xV

                                                                                                                            V

                                                                                                                            VxPV vxVv

                                                                                                                            min

                                                                                                                            xPV

                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            d VB

                                                                                                                            xBBBBxP tVV

                                                                                                                            tVVV

                                                                                                                            1

                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                            (thus projection is another linear operation)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            d VB

                                                                                                                            xBBBBxP tVV

                                                                                                                            tVVV

                                                                                                                            1

                                                                                                                            tVV

                                                                                                                            tVVV BBBBP

                                                                                                                            1

                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                            (thus projection is another linear operation)

                                                                                                                            (note same operation underlies least squares)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            d VB

                                                                                                                            xBBBBxP tVV

                                                                                                                            tVVV

                                                                                                                            1

                                                                                                                            tVV

                                                                                                                            tVVV BBBBP

                                                                                                                            1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Projection using Orthonormal Basis

                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                            nnVtV IBB

                                                                                                                            10

                                                                                                                            01

                                                                                                                            1

                                                                                                                            111

                                                                                                                            1

                                                                                                                            1

                                                                                                                            nnn

                                                                                                                            n

                                                                                                                            ntn

                                                                                                                            t

                                                                                                                            vvvv

                                                                                                                            vvvv

                                                                                                                            vv

                                                                                                                            v

                                                                                                                            v

                                                                                                                            nvv 1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Projection using Orthonormal Basis

                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                            bull So =

                                                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                            (Recall Right Multrsquon)

                                                                                                                            nnVtV IBB

                                                                                                                            10

                                                                                                                            01

                                                                                                                            1

                                                                                                                            111

                                                                                                                            1

                                                                                                                            1

                                                                                                                            nnn

                                                                                                                            n

                                                                                                                            ntn

                                                                                                                            t

                                                                                                                            vvvv

                                                                                                                            vvvv

                                                                                                                            vv

                                                                                                                            v

                                                                                                                            v

                                                                                                                            xBBxP tVVV

                                                                                                                            x V

                                                                                                                            nvv 1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                            bull For Orthogonal Complement

                                                                                                                            and

                                                                                                                            V

                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                            bull For Orthogonal Complement

                                                                                                                            and

                                                                                                                            bull Parseval Inequality

                                                                                                                            V

                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                            2

                                                                                                                            1

                                                                                                                            22

                                                                                                                            1

                                                                                                                            22 aavxxxP

                                                                                                                            n

                                                                                                                            ii

                                                                                                                            n

                                                                                                                            iiV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            (Real) Unitary Matrices with

                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                            (So All of Above Applies)

                                                                                                                            ddU IUU t

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            (Real) Unitary Matrices with

                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                            (So All of Above Applies)

                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                            ddU IUU t

                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                            i ii 2

                                                                                                                            1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            (Real) Unitary Matrices with

                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                            (So All of Above Applies)

                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                            ddU IUU t

                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                            i ii 2

                                                                                                                            1

                                                                                                                            U

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                            For a Matrix

                                                                                                                            Find

                                                                                                                            ndX

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                            For a Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            with Entries

                                                                                                                            called Singular Values

                                                                                                                            ndX

                                                                                                                            ndS

                                                                                                                            )min(1 ndss

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                            For a Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            with Entries

                                                                                                                            called Singular Values

                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                            (recall )

                                                                                                                            ndX

                                                                                                                            ndS

                                                                                                                            )min(1 ndss

                                                                                                                            ddU nnV

                                                                                                                            IVVUU tt

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                            For a Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            with Entries

                                                                                                                            called Singular Values

                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                            (recall )

                                                                                                                            So That

                                                                                                                            ndX

                                                                                                                            ndS

                                                                                                                            )min(1 ndss

                                                                                                                            ddU nnV

                                                                                                                            IVVUU tt tUSVX

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                            vVSUvVSUvX tt

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                            bull First rotate

                                                                                                                            X

                                                                                                                            vVSUvVSUvX tt

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                            bull First rotate

                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                            X

                                                                                                                            vVSUvVSUvX tt

                                                                                                                            is

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                            bull First rotate

                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                            bull Third rotate again

                                                                                                                            X

                                                                                                                            vVSUvVSUvX tt

                                                                                                                            is

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                            bull First rotate

                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                            bull Third rotate again

                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                            X

                                                                                                                            vVSUvVSUvX tt

                                                                                                                            is

                                                                                                                            SVD Compact Representation

                                                                                                                            Useful Labeling

                                                                                                                            Singular Values in Increasing Order

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            )min(1 dnss

                                                                                                                            SVD Compact Representation

                                                                                                                            Useful Labeling

                                                                                                                            Singular Values in Increasing Order

                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            )min(1 dnss

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            r

                                                                                                                            SVD Compact Representation

                                                                                                                            Useful Labeling

                                                                                                                            Singular Values in Increasing Order

                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                            Let = of positive singular values

                                                                                                                            )min(1 dnss

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            r

                                                                                                                            SVD Compact Representation

                                                                                                                            Useful Labeling

                                                                                                                            Singular Values in Increasing Order

                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                            Let = of positive singular values

                                                                                                                            Then

                                                                                                                            Where are truncations of

                                                                                                                            trnrrrd VSUX

                                                                                                                            VSU

                                                                                                                            )min(1 dnss

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Full Representation

                                                                                                                            =

                                                                                                                            Graphics Display Assumes

                                                                                                                            ndX ddU ndS nn

                                                                                                                            tV

                                                                                                                            nd

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Full Representation

                                                                                                                            =

                                                                                                                            Full Rank Basis Matrix

                                                                                                                            ndX ddU ndS nn

                                                                                                                            tV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Full Representation

                                                                                                                            =

                                                                                                                            Full Rank Basis Matrix

                                                                                                                            All 0s in Bottom

                                                                                                                            ndX ddU ndS nn

                                                                                                                            tV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Reduced Representation

                                                                                                                            =

                                                                                                                            These Columns Get 0ed Out

                                                                                                                            ndX ddU nnS nn

                                                                                                                            tV

                                                                                                                            nnd 0

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Reduced Representation

                                                                                                                            =ndX ndU nnS nn

                                                                                                                            tV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Reduced Representation

                                                                                                                            =

                                                                                                                            Also Some of These May be 0

                                                                                                                            ndX ndU nnS nn

                                                                                                                            tV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Compact Representation

                                                                                                                            = ndX rdU

                                                                                                                            rrS nrtV

                                                                                                                            0

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Compact Representation

                                                                                                                            =

                                                                                                                            These Get 0ed Out

                                                                                                                            ndX rdU

                                                                                                                            rrS nrtV

                                                                                                                            0

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            SVD Compact Representation

                                                                                                                            = ndX rdU

                                                                                                                            rrS nrtV

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition

                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                            Find

                                                                                                                            ddX

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition

                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            Called Eigenvalues

                                                                                                                            Convenient Ordering

                                                                                                                            ddX

                                                                                                                            d

                                                                                                                            D

                                                                                                                            0

                                                                                                                            01

                                                                                                                            n 1

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition

                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            And an Orthonormal Matrix

                                                                                                                            (ie )

                                                                                                                            ddX

                                                                                                                            d

                                                                                                                            D

                                                                                                                            0

                                                                                                                            01

                                                                                                                            ddB

                                                                                                                            ddtt IBBBB

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition

                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                            Find a Diagonal Matrix

                                                                                                                            And an Orthonormal Matrix

                                                                                                                            (ie )

                                                                                                                            So that ie

                                                                                                                            ddX

                                                                                                                            d

                                                                                                                            D

                                                                                                                            0

                                                                                                                            01

                                                                                                                            ddB

                                                                                                                            ddtt IBBBB

                                                                                                                            DBBX tBDBX

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                            (looks similar)

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                            Complex (uses )

                                                                                                                            VU

                                                                                                                            1i

                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                            VU

                                                                                                                            X

                                                                                                                            BVU

                                                                                                                            1i

                                                                                                                            • Participant Presentations
                                                                                                                            • PCA to find clusters
                                                                                                                            • Statistical Smoothing
                                                                                                                            • Kernel Density Estimation
                                                                                                                            • Scatterplot Smoothing
                                                                                                                            • Statistical Smoothing (2)
                                                                                                                            • SiZer Background
                                                                                                                            • SiZer Background (2)
                                                                                                                            • SiZer Background (3)
                                                                                                                            • SiZer Background (4)
                                                                                                                            • SiZer Background (5)
                                                                                                                            • SiZer Overview
                                                                                                                            • PCA to find clusters (2)
                                                                                                                            • PCA to find clusters (3)
                                                                                                                            • PCA to find clusters (4)
                                                                                                                            • PCA to find clusters (5)
                                                                                                                            • PCA to find clusters (6)
                                                                                                                            • PCA to find clusters (7)
                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                            • Frequency 2 Analysis
                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                            • Yeast Cell Cycle
                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                            • Reclassification of Major Genes
                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                            • New Subpopulation View
                                                                                                                            • New Subpopulation View (2)
                                                                                                                            • Detailed Look at PCA
                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                            • An Interesting Historical Note
                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                            • Review of Linear Algebra
                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                                              bull (Use Basis Above)

                                                                                                                              bull eg dim of a line is 1

                                                                                                                              bull eg dim of a plane is 2

                                                                                                                              dd dim

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                              bull Number of Elements in a Basis (Unique)

                                                                                                                              bull (Use Basis Above)

                                                                                                                              bull eg dim of a line is 1

                                                                                                                              bull eg dim of a plane is 2

                                                                                                                              bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                              (in Statistical Uses eg ANOVA)

                                                                                                                              dd dim

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Norm of a Vector

                                                                                                                              bull in d 21

                                                                                                                              21

                                                                                                                              1

                                                                                                                              2 xxxx td

                                                                                                                              jj

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Norm of a Vector

                                                                                                                              bull in

                                                                                                                              bull Idea length of the vector

                                                                                                                              d 2121

                                                                                                                              1

                                                                                                                              2 xxxx td

                                                                                                                              jj

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Norm of a Vector

                                                                                                                              bull in

                                                                                                                              bull Idea length of the vector

                                                                                                                              bull Note strange properties for high

                                                                                                                              eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                              d 2121

                                                                                                                              1

                                                                                                                              2 xxxx td

                                                                                                                              jj

                                                                                                                              d

                                                                                                                              d

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Norm of a Vector (cont)

                                                                                                                              bull Length Normalized Vector

                                                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                                                              amp is a Direction Vector)

                                                                                                                              x

                                                                                                                              x

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Norm of a Vector (cont)

                                                                                                                              bull Length Normalized Vector

                                                                                                                              (has Length 1 thus on Surf of Unit Sphere

                                                                                                                              amp is a Direction Vector)

                                                                                                                              bull Define Distance as

                                                                                                                              x

                                                                                                                              x

                                                                                                                              yxyxyxyxd t

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Inner (Dot Scalar) Product

                                                                                                                              bull for Vectors and

                                                                                                                              yxyxyx td

                                                                                                                              jjj

                                                                                                                              1

                                                                                                                              x y

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Inner (Dot Scalar) Product

                                                                                                                              bull for Vectors and

                                                                                                                              bull Related to Norm via

                                                                                                                              yxyxyx td

                                                                                                                              jjj

                                                                                                                              1

                                                                                                                              x y

                                                                                                                              21 xxx

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                                              yyxx

                                                                                                                              yx

                                                                                                                              yx

                                                                                                                              yxyxangle

                                                                                                                              tt

                                                                                                                              t

                                                                                                                              11 cos

                                                                                                                              cos

                                                                                                                              x y

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                              if and only if

                                                                                                                              yyxx

                                                                                                                              yx

                                                                                                                              yx

                                                                                                                              yxyxangle

                                                                                                                              tt

                                                                                                                              t

                                                                                                                              11 cos

                                                                                                                              cos

                                                                                                                              x y

                                                                                                                              yx 0 yx

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis

                                                                                                                              bull All Orthogonal to each other

                                                                                                                              ie for

                                                                                                                              bull All have Length 1

                                                                                                                              ie for

                                                                                                                              nvv 1

                                                                                                                              1 ii vv

                                                                                                                              0 ii vv ii

                                                                                                                              ni 1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis (cont)

                                                                                                                              bull Spectral Representation

                                                                                                                              where

                                                                                                                              nvv 1

                                                                                                                              n

                                                                                                                              iii vax

                                                                                                                              1

                                                                                                                              ii vxa

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis (cont)

                                                                                                                              bull Spectral Representation

                                                                                                                              where

                                                                                                                              (Coefficient is Inner Product Cool Notation)

                                                                                                                              nvv 1

                                                                                                                              n

                                                                                                                              iii vax

                                                                                                                              1

                                                                                                                              ii vxa

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis (cont)

                                                                                                                              bull Spectral Representation

                                                                                                                              where

                                                                                                                              Check

                                                                                                                              nvv 1

                                                                                                                              n

                                                                                                                              iii vax

                                                                                                                              1

                                                                                                                              ii vxa

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iiii avvavvavx

                                                                                                                              1

                                                                                                                              1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis (cont)

                                                                                                                              bull Spectral Representation

                                                                                                                              where

                                                                                                                              Check

                                                                                                                              bull Matrix Notation where ie

                                                                                                                              For the Basis Matrix

                                                                                                                              nvv 1

                                                                                                                              n

                                                                                                                              iii vax

                                                                                                                              1

                                                                                                                              ii vxa

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iiii avvavvavx

                                                                                                                              1

                                                                                                                              1

                                                                                                                              aBx Bxa tt xBa t

                                                                                                                              nvvB 1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Orthonormal Basis (cont)

                                                                                                                              bull Spectral Representation

                                                                                                                              where

                                                                                                                              Check

                                                                                                                              bull Matrix Notation where ie

                                                                                                                              is called transform of

                                                                                                                              (eg Fourier or Wavelet)

                                                                                                                              nvv 1

                                                                                                                              n

                                                                                                                              iii vax

                                                                                                                              1

                                                                                                                              ii vxa

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iii

                                                                                                                              n

                                                                                                                              iiii avvavvavx

                                                                                                                              1

                                                                                                                              1

                                                                                                                              aBx Bxa tt xBa t

                                                                                                                              xa

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Parseval identity for

                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                              x

                                                                                                                              nvv 1

                                                                                                                              2

                                                                                                                              1

                                                                                                                              22

                                                                                                                              1

                                                                                                                              2 aavxx

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Parseval identity for

                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                              bull Pythagorean theorem

                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                              bull ANOVA - sums of squares

                                                                                                                              x

                                                                                                                              nvv 1

                                                                                                                              2

                                                                                                                              1

                                                                                                                              22

                                                                                                                              1

                                                                                                                              2 aavxx

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Parseval identity for

                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                              bull Pythagorean theorem

                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                              bull ANOVA - sums of squares

                                                                                                                              bull Transform has same length as

                                                                                                                              ie ldquorotation in rdquo

                                                                                                                              x

                                                                                                                              nvv 1

                                                                                                                              2

                                                                                                                              1

                                                                                                                              22

                                                                                                                              1

                                                                                                                              2 aavxx

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              a xd

                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                              xV

                                                                                                                              V

                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                              bull Find that Solves

                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                              xV

                                                                                                                              V

                                                                                                                              VxPV vxVv

                                                                                                                              min

                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                              bull Find that Solves

                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                              bull For Inner Product (Hilbert) Space

                                                                                                                              Exists and is Unique

                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                              xV

                                                                                                                              V

                                                                                                                              VxPV vxVv

                                                                                                                              min

                                                                                                                              xPV

                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              d VB

                                                                                                                              xBBBBxP tVV

                                                                                                                              tVVV

                                                                                                                              1

                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                              (thus projection is another linear operation)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              d VB

                                                                                                                              xBBBBxP tVV

                                                                                                                              tVVV

                                                                                                                              1

                                                                                                                              tVV

                                                                                                                              tVVV BBBBP

                                                                                                                              1

                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                              (thus projection is another linear operation)

                                                                                                                              (note same operation underlies least squares)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              d VB

                                                                                                                              xBBBBxP tVV

                                                                                                                              tVVV

                                                                                                                              1

                                                                                                                              tVV

                                                                                                                              tVVV BBBBP

                                                                                                                              1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Projection using Orthonormal Basis

                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                              nnVtV IBB

                                                                                                                              10

                                                                                                                              01

                                                                                                                              1

                                                                                                                              111

                                                                                                                              1

                                                                                                                              1

                                                                                                                              nnn

                                                                                                                              n

                                                                                                                              ntn

                                                                                                                              t

                                                                                                                              vvvv

                                                                                                                              vvvv

                                                                                                                              vv

                                                                                                                              v

                                                                                                                              v

                                                                                                                              nvv 1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Projection using Orthonormal Basis

                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                              bull So =

                                                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                              (Recall Right Multrsquon)

                                                                                                                              nnVtV IBB

                                                                                                                              10

                                                                                                                              01

                                                                                                                              1

                                                                                                                              111

                                                                                                                              1

                                                                                                                              1

                                                                                                                              nnn

                                                                                                                              n

                                                                                                                              ntn

                                                                                                                              t

                                                                                                                              vvvv

                                                                                                                              vvvv

                                                                                                                              vv

                                                                                                                              v

                                                                                                                              v

                                                                                                                              xBBxP tVVV

                                                                                                                              x V

                                                                                                                              nvv 1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                              bull For Orthogonal Complement

                                                                                                                              and

                                                                                                                              V

                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                              bull For Orthogonal Complement

                                                                                                                              and

                                                                                                                              bull Parseval Inequality

                                                                                                                              V

                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                              2

                                                                                                                              1

                                                                                                                              22

                                                                                                                              1

                                                                                                                              22 aavxxxP

                                                                                                                              n

                                                                                                                              ii

                                                                                                                              n

                                                                                                                              iiV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              (Real) Unitary Matrices with

                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                              (So All of Above Applies)

                                                                                                                              ddU IUU t

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              (Real) Unitary Matrices with

                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                              (So All of Above Applies)

                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                              ddU IUU t

                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                              i ii 2

                                                                                                                              1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              (Real) Unitary Matrices with

                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                              (So All of Above Applies)

                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                              ddU IUU t

                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                              i ii 2

                                                                                                                              1

                                                                                                                              U

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                              For a Matrix

                                                                                                                              Find

                                                                                                                              ndX

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                              For a Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              with Entries

                                                                                                                              called Singular Values

                                                                                                                              ndX

                                                                                                                              ndS

                                                                                                                              )min(1 ndss

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                              For a Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              with Entries

                                                                                                                              called Singular Values

                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                              (recall )

                                                                                                                              ndX

                                                                                                                              ndS

                                                                                                                              )min(1 ndss

                                                                                                                              ddU nnV

                                                                                                                              IVVUU tt

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                              For a Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              with Entries

                                                                                                                              called Singular Values

                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                              (recall )

                                                                                                                              So That

                                                                                                                              ndX

                                                                                                                              ndS

                                                                                                                              )min(1 ndss

                                                                                                                              ddU nnV

                                                                                                                              IVVUU tt tUSVX

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                              vVSUvVSUvX tt

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                              bull First rotate

                                                                                                                              X

                                                                                                                              vVSUvVSUvX tt

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                              bull First rotate

                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                              X

                                                                                                                              vVSUvVSUvX tt

                                                                                                                              is

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                              bull First rotate

                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                              bull Third rotate again

                                                                                                                              X

                                                                                                                              vVSUvVSUvX tt

                                                                                                                              is

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                              bull First rotate

                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                              bull Third rotate again

                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                              X

                                                                                                                              vVSUvVSUvX tt

                                                                                                                              is

                                                                                                                              SVD Compact Representation

                                                                                                                              Useful Labeling

                                                                                                                              Singular Values in Increasing Order

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              )min(1 dnss

                                                                                                                              SVD Compact Representation

                                                                                                                              Useful Labeling

                                                                                                                              Singular Values in Increasing Order

                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              )min(1 dnss

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              r

                                                                                                                              SVD Compact Representation

                                                                                                                              Useful Labeling

                                                                                                                              Singular Values in Increasing Order

                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                              Let = of positive singular values

                                                                                                                              )min(1 dnss

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              r

                                                                                                                              SVD Compact Representation

                                                                                                                              Useful Labeling

                                                                                                                              Singular Values in Increasing Order

                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                              Let = of positive singular values

                                                                                                                              Then

                                                                                                                              Where are truncations of

                                                                                                                              trnrrrd VSUX

                                                                                                                              VSU

                                                                                                                              )min(1 dnss

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Full Representation

                                                                                                                              =

                                                                                                                              Graphics Display Assumes

                                                                                                                              ndX ddU ndS nn

                                                                                                                              tV

                                                                                                                              nd

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Full Representation

                                                                                                                              =

                                                                                                                              Full Rank Basis Matrix

                                                                                                                              ndX ddU ndS nn

                                                                                                                              tV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Full Representation

                                                                                                                              =

                                                                                                                              Full Rank Basis Matrix

                                                                                                                              All 0s in Bottom

                                                                                                                              ndX ddU ndS nn

                                                                                                                              tV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Reduced Representation

                                                                                                                              =

                                                                                                                              These Columns Get 0ed Out

                                                                                                                              ndX ddU nnS nn

                                                                                                                              tV

                                                                                                                              nnd 0

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Reduced Representation

                                                                                                                              =ndX ndU nnS nn

                                                                                                                              tV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Reduced Representation

                                                                                                                              =

                                                                                                                              Also Some of These May be 0

                                                                                                                              ndX ndU nnS nn

                                                                                                                              tV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Compact Representation

                                                                                                                              = ndX rdU

                                                                                                                              rrS nrtV

                                                                                                                              0

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Compact Representation

                                                                                                                              =

                                                                                                                              These Get 0ed Out

                                                                                                                              ndX rdU

                                                                                                                              rrS nrtV

                                                                                                                              0

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              SVD Compact Representation

                                                                                                                              = ndX rdU

                                                                                                                              rrS nrtV

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition

                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                              Find

                                                                                                                              ddX

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition

                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              Called Eigenvalues

                                                                                                                              Convenient Ordering

                                                                                                                              ddX

                                                                                                                              d

                                                                                                                              D

                                                                                                                              0

                                                                                                                              01

                                                                                                                              n 1

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition

                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              And an Orthonormal Matrix

                                                                                                                              (ie )

                                                                                                                              ddX

                                                                                                                              d

                                                                                                                              D

                                                                                                                              0

                                                                                                                              01

                                                                                                                              ddB

                                                                                                                              ddtt IBBBB

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition

                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                              Find a Diagonal Matrix

                                                                                                                              And an Orthonormal Matrix

                                                                                                                              (ie )

                                                                                                                              So that ie

                                                                                                                              ddX

                                                                                                                              d

                                                                                                                              D

                                                                                                                              0

                                                                                                                              01

                                                                                                                              ddB

                                                                                                                              ddtt IBBBB

                                                                                                                              DBBX tBDBX

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                              (looks similar)

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                              Complex (uses )

                                                                                                                              VU

                                                                                                                              1i

                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                              VU

                                                                                                                              X

                                                                                                                              BVU

                                                                                                                              1i

                                                                                                                              • Participant Presentations
                                                                                                                              • PCA to find clusters
                                                                                                                              • Statistical Smoothing
                                                                                                                              • Kernel Density Estimation
                                                                                                                              • Scatterplot Smoothing
                                                                                                                              • Statistical Smoothing (2)
                                                                                                                              • SiZer Background
                                                                                                                              • SiZer Background (2)
                                                                                                                              • SiZer Background (3)
                                                                                                                              • SiZer Background (4)
                                                                                                                              • SiZer Background (5)
                                                                                                                              • SiZer Overview
                                                                                                                              • PCA to find clusters (2)
                                                                                                                              • PCA to find clusters (3)
                                                                                                                              • PCA to find clusters (4)
                                                                                                                              • PCA to find clusters (5)
                                                                                                                              • PCA to find clusters (6)
                                                                                                                              • PCA to find clusters (7)
                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                              • Frequency 2 Analysis
                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                              • Yeast Cell Cycle
                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                              • Reclassification of Major Genes
                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                              • New Subpopulation View
                                                                                                                              • New Subpopulation View (2)
                                                                                                                              • Detailed Look at PCA
                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                              • An Interesting Historical Note
                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                              • Review of Linear Algebra
                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                                                bull (Use Basis Above)

                                                                                                                                bull eg dim of a line is 1

                                                                                                                                bull eg dim of a plane is 2

                                                                                                                                dd dim

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                                bull Number of Elements in a Basis (Unique)

                                                                                                                                bull (Use Basis Above)

                                                                                                                                bull eg dim of a line is 1

                                                                                                                                bull eg dim of a plane is 2

                                                                                                                                bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                                (in Statistical Uses eg ANOVA)

                                                                                                                                dd dim

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Norm of a Vector

                                                                                                                                bull in d 21

                                                                                                                                21

                                                                                                                                1

                                                                                                                                2 xxxx td

                                                                                                                                jj

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Norm of a Vector

                                                                                                                                bull in

                                                                                                                                bull Idea length of the vector

                                                                                                                                d 2121

                                                                                                                                1

                                                                                                                                2 xxxx td

                                                                                                                                jj

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Norm of a Vector

                                                                                                                                bull in

                                                                                                                                bull Idea length of the vector

                                                                                                                                bull Note strange properties for high

                                                                                                                                eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                                d 2121

                                                                                                                                1

                                                                                                                                2 xxxx td

                                                                                                                                jj

                                                                                                                                d

                                                                                                                                d

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Norm of a Vector (cont)

                                                                                                                                bull Length Normalized Vector

                                                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                amp is a Direction Vector)

                                                                                                                                x

                                                                                                                                x

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Norm of a Vector (cont)

                                                                                                                                bull Length Normalized Vector

                                                                                                                                (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                amp is a Direction Vector)

                                                                                                                                bull Define Distance as

                                                                                                                                x

                                                                                                                                x

                                                                                                                                yxyxyxyxd t

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Inner (Dot Scalar) Product

                                                                                                                                bull for Vectors and

                                                                                                                                yxyxyx td

                                                                                                                                jjj

                                                                                                                                1

                                                                                                                                x y

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Inner (Dot Scalar) Product

                                                                                                                                bull for Vectors and

                                                                                                                                bull Related to Norm via

                                                                                                                                yxyxyx td

                                                                                                                                jjj

                                                                                                                                1

                                                                                                                                x y

                                                                                                                                21 xxx

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                                yyxx

                                                                                                                                yx

                                                                                                                                yx

                                                                                                                                yxyxangle

                                                                                                                                tt

                                                                                                                                t

                                                                                                                                11 cos

                                                                                                                                cos

                                                                                                                                x y

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                if and only if

                                                                                                                                yyxx

                                                                                                                                yx

                                                                                                                                yx

                                                                                                                                yxyxangle

                                                                                                                                tt

                                                                                                                                t

                                                                                                                                11 cos

                                                                                                                                cos

                                                                                                                                x y

                                                                                                                                yx 0 yx

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis

                                                                                                                                bull All Orthogonal to each other

                                                                                                                                ie for

                                                                                                                                bull All have Length 1

                                                                                                                                ie for

                                                                                                                                nvv 1

                                                                                                                                1 ii vv

                                                                                                                                0 ii vv ii

                                                                                                                                ni 1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                bull Spectral Representation

                                                                                                                                where

                                                                                                                                nvv 1

                                                                                                                                n

                                                                                                                                iii vax

                                                                                                                                1

                                                                                                                                ii vxa

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                bull Spectral Representation

                                                                                                                                where

                                                                                                                                (Coefficient is Inner Product Cool Notation)

                                                                                                                                nvv 1

                                                                                                                                n

                                                                                                                                iii vax

                                                                                                                                1

                                                                                                                                ii vxa

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                bull Spectral Representation

                                                                                                                                where

                                                                                                                                Check

                                                                                                                                nvv 1

                                                                                                                                n

                                                                                                                                iii vax

                                                                                                                                1

                                                                                                                                ii vxa

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iiii avvavvavx

                                                                                                                                1

                                                                                                                                1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                bull Spectral Representation

                                                                                                                                where

                                                                                                                                Check

                                                                                                                                bull Matrix Notation where ie

                                                                                                                                For the Basis Matrix

                                                                                                                                nvv 1

                                                                                                                                n

                                                                                                                                iii vax

                                                                                                                                1

                                                                                                                                ii vxa

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iiii avvavvavx

                                                                                                                                1

                                                                                                                                1

                                                                                                                                aBx Bxa tt xBa t

                                                                                                                                nvvB 1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                bull Spectral Representation

                                                                                                                                where

                                                                                                                                Check

                                                                                                                                bull Matrix Notation where ie

                                                                                                                                is called transform of

                                                                                                                                (eg Fourier or Wavelet)

                                                                                                                                nvv 1

                                                                                                                                n

                                                                                                                                iii vax

                                                                                                                                1

                                                                                                                                ii vxa

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iii

                                                                                                                                n

                                                                                                                                iiii avvavvavx

                                                                                                                                1

                                                                                                                                1

                                                                                                                                aBx Bxa tt xBa t

                                                                                                                                xa

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Parseval identity for

                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                x

                                                                                                                                nvv 1

                                                                                                                                2

                                                                                                                                1

                                                                                                                                22

                                                                                                                                1

                                                                                                                                2 aavxx

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Parseval identity for

                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                bull Pythagorean theorem

                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                x

                                                                                                                                nvv 1

                                                                                                                                2

                                                                                                                                1

                                                                                                                                22

                                                                                                                                1

                                                                                                                                2 aavxx

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Parseval identity for

                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                bull Pythagorean theorem

                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                bull Transform has same length as

                                                                                                                                ie ldquorotation in rdquo

                                                                                                                                x

                                                                                                                                nvv 1

                                                                                                                                2

                                                                                                                                1

                                                                                                                                22

                                                                                                                                1

                                                                                                                                2 aavxx

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                a xd

                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                xV

                                                                                                                                V

                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                bull Find that Solves

                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                xV

                                                                                                                                V

                                                                                                                                VxPV vxVv

                                                                                                                                min

                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                bull Find that Solves

                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                bull For Inner Product (Hilbert) Space

                                                                                                                                Exists and is Unique

                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                xV

                                                                                                                                V

                                                                                                                                VxPV vxVv

                                                                                                                                min

                                                                                                                                xPV

                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                d VB

                                                                                                                                xBBBBxP tVV

                                                                                                                                tVVV

                                                                                                                                1

                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                (thus projection is another linear operation)

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                d VB

                                                                                                                                xBBBBxP tVV

                                                                                                                                tVVV

                                                                                                                                1

                                                                                                                                tVV

                                                                                                                                tVVV BBBBP

                                                                                                                                1

                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                (thus projection is another linear operation)

                                                                                                                                (note same operation underlies least squares)

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                d VB

                                                                                                                                xBBBBxP tVV

                                                                                                                                tVVV

                                                                                                                                1

                                                                                                                                tVV

                                                                                                                                tVVV BBBBP

                                                                                                                                1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                nnVtV IBB

                                                                                                                                10

                                                                                                                                01

                                                                                                                                1

                                                                                                                                111

                                                                                                                                1

                                                                                                                                1

                                                                                                                                nnn

                                                                                                                                n

                                                                                                                                ntn

                                                                                                                                t

                                                                                                                                vvvv

                                                                                                                                vvvv

                                                                                                                                vv

                                                                                                                                v

                                                                                                                                v

                                                                                                                                nvv 1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                bull So =

                                                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                (Recall Right Multrsquon)

                                                                                                                                nnVtV IBB

                                                                                                                                10

                                                                                                                                01

                                                                                                                                1

                                                                                                                                111

                                                                                                                                1

                                                                                                                                1

                                                                                                                                nnn

                                                                                                                                n

                                                                                                                                ntn

                                                                                                                                t

                                                                                                                                vvvv

                                                                                                                                vvvv

                                                                                                                                vv

                                                                                                                                v

                                                                                                                                v

                                                                                                                                xBBxP tVVV

                                                                                                                                x V

                                                                                                                                nvv 1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                bull For Orthogonal Complement

                                                                                                                                and

                                                                                                                                V

                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                bull For Orthogonal Complement

                                                                                                                                and

                                                                                                                                bull Parseval Inequality

                                                                                                                                V

                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                2

                                                                                                                                1

                                                                                                                                22

                                                                                                                                1

                                                                                                                                22 aavxxxP

                                                                                                                                n

                                                                                                                                ii

                                                                                                                                n

                                                                                                                                iiV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                (So All of Above Applies)

                                                                                                                                ddU IUU t

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                (So All of Above Applies)

                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                ddU IUU t

                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                i ii 2

                                                                                                                                1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                (So All of Above Applies)

                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                ddU IUU t

                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                i ii 2

                                                                                                                                1

                                                                                                                                U

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                For a Matrix

                                                                                                                                Find

                                                                                                                                ndX

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                For a Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                with Entries

                                                                                                                                called Singular Values

                                                                                                                                ndX

                                                                                                                                ndS

                                                                                                                                )min(1 ndss

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                For a Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                with Entries

                                                                                                                                called Singular Values

                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                (recall )

                                                                                                                                ndX

                                                                                                                                ndS

                                                                                                                                )min(1 ndss

                                                                                                                                ddU nnV

                                                                                                                                IVVUU tt

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                For a Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                with Entries

                                                                                                                                called Singular Values

                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                (recall )

                                                                                                                                So That

                                                                                                                                ndX

                                                                                                                                ndS

                                                                                                                                )min(1 ndss

                                                                                                                                ddU nnV

                                                                                                                                IVVUU tt tUSVX

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                vVSUvVSUvX tt

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                bull First rotate

                                                                                                                                X

                                                                                                                                vVSUvVSUvX tt

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                bull First rotate

                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                X

                                                                                                                                vVSUvVSUvX tt

                                                                                                                                is

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                bull First rotate

                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                bull Third rotate again

                                                                                                                                X

                                                                                                                                vVSUvVSUvX tt

                                                                                                                                is

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                bull First rotate

                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                bull Third rotate again

                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                X

                                                                                                                                vVSUvVSUvX tt

                                                                                                                                is

                                                                                                                                SVD Compact Representation

                                                                                                                                Useful Labeling

                                                                                                                                Singular Values in Increasing Order

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                )min(1 dnss

                                                                                                                                SVD Compact Representation

                                                                                                                                Useful Labeling

                                                                                                                                Singular Values in Increasing Order

                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                )min(1 dnss

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                r

                                                                                                                                SVD Compact Representation

                                                                                                                                Useful Labeling

                                                                                                                                Singular Values in Increasing Order

                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                Let = of positive singular values

                                                                                                                                )min(1 dnss

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                r

                                                                                                                                SVD Compact Representation

                                                                                                                                Useful Labeling

                                                                                                                                Singular Values in Increasing Order

                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                Let = of positive singular values

                                                                                                                                Then

                                                                                                                                Where are truncations of

                                                                                                                                trnrrrd VSUX

                                                                                                                                VSU

                                                                                                                                )min(1 dnss

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Full Representation

                                                                                                                                =

                                                                                                                                Graphics Display Assumes

                                                                                                                                ndX ddU ndS nn

                                                                                                                                tV

                                                                                                                                nd

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Full Representation

                                                                                                                                =

                                                                                                                                Full Rank Basis Matrix

                                                                                                                                ndX ddU ndS nn

                                                                                                                                tV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Full Representation

                                                                                                                                =

                                                                                                                                Full Rank Basis Matrix

                                                                                                                                All 0s in Bottom

                                                                                                                                ndX ddU ndS nn

                                                                                                                                tV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Reduced Representation

                                                                                                                                =

                                                                                                                                These Columns Get 0ed Out

                                                                                                                                ndX ddU nnS nn

                                                                                                                                tV

                                                                                                                                nnd 0

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Reduced Representation

                                                                                                                                =ndX ndU nnS nn

                                                                                                                                tV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Reduced Representation

                                                                                                                                =

                                                                                                                                Also Some of These May be 0

                                                                                                                                ndX ndU nnS nn

                                                                                                                                tV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Compact Representation

                                                                                                                                = ndX rdU

                                                                                                                                rrS nrtV

                                                                                                                                0

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Compact Representation

                                                                                                                                =

                                                                                                                                These Get 0ed Out

                                                                                                                                ndX rdU

                                                                                                                                rrS nrtV

                                                                                                                                0

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                SVD Compact Representation

                                                                                                                                = ndX rdU

                                                                                                                                rrS nrtV

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition

                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                Find

                                                                                                                                ddX

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition

                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                Called Eigenvalues

                                                                                                                                Convenient Ordering

                                                                                                                                ddX

                                                                                                                                d

                                                                                                                                D

                                                                                                                                0

                                                                                                                                01

                                                                                                                                n 1

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition

                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                And an Orthonormal Matrix

                                                                                                                                (ie )

                                                                                                                                ddX

                                                                                                                                d

                                                                                                                                D

                                                                                                                                0

                                                                                                                                01

                                                                                                                                ddB

                                                                                                                                ddtt IBBBB

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition

                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                Find a Diagonal Matrix

                                                                                                                                And an Orthonormal Matrix

                                                                                                                                (ie )

                                                                                                                                So that ie

                                                                                                                                ddX

                                                                                                                                d

                                                                                                                                D

                                                                                                                                0

                                                                                                                                01

                                                                                                                                ddB

                                                                                                                                ddtt IBBBB

                                                                                                                                DBBX tBDBX

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                (looks similar)

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                Complex (uses )

                                                                                                                                VU

                                                                                                                                1i

                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                VU

                                                                                                                                X

                                                                                                                                BVU

                                                                                                                                1i

                                                                                                                                • Participant Presentations
                                                                                                                                • PCA to find clusters
                                                                                                                                • Statistical Smoothing
                                                                                                                                • Kernel Density Estimation
                                                                                                                                • Scatterplot Smoothing
                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                • SiZer Background
                                                                                                                                • SiZer Background (2)
                                                                                                                                • SiZer Background (3)
                                                                                                                                • SiZer Background (4)
                                                                                                                                • SiZer Background (5)
                                                                                                                                • SiZer Overview
                                                                                                                                • PCA to find clusters (2)
                                                                                                                                • PCA to find clusters (3)
                                                                                                                                • PCA to find clusters (4)
                                                                                                                                • PCA to find clusters (5)
                                                                                                                                • PCA to find clusters (6)
                                                                                                                                • PCA to find clusters (7)
                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                • Frequency 2 Analysis
                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                • Yeast Cell Cycle
                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                • Reclassification of Major Genes
                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                • New Subpopulation View
                                                                                                                                • New Subpopulation View (2)
                                                                                                                                • Detailed Look at PCA
                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                • An Interesting Historical Note
                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                • Review of Linear Algebra
                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Dimension of Subspace (a Notion of ldquoSizerdquo)

                                                                                                                                  bull Number of Elements in a Basis (Unique)

                                                                                                                                  bull (Use Basis Above)

                                                                                                                                  bull eg dim of a line is 1

                                                                                                                                  bull eg dim of a plane is 2

                                                                                                                                  bull Dimension is ldquoDegrees of Freedomrdquo

                                                                                                                                  (in Statistical Uses eg ANOVA)

                                                                                                                                  dd dim

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Norm of a Vector

                                                                                                                                  bull in d 21

                                                                                                                                  21

                                                                                                                                  1

                                                                                                                                  2 xxxx td

                                                                                                                                  jj

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Norm of a Vector

                                                                                                                                  bull in

                                                                                                                                  bull Idea length of the vector

                                                                                                                                  d 2121

                                                                                                                                  1

                                                                                                                                  2 xxxx td

                                                                                                                                  jj

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Norm of a Vector

                                                                                                                                  bull in

                                                                                                                                  bull Idea length of the vector

                                                                                                                                  bull Note strange properties for high

                                                                                                                                  eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                                  d 2121

                                                                                                                                  1

                                                                                                                                  2 xxxx td

                                                                                                                                  jj

                                                                                                                                  d

                                                                                                                                  d

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Norm of a Vector (cont)

                                                                                                                                  bull Length Normalized Vector

                                                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                  amp is a Direction Vector)

                                                                                                                                  x

                                                                                                                                  x

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Norm of a Vector (cont)

                                                                                                                                  bull Length Normalized Vector

                                                                                                                                  (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                  amp is a Direction Vector)

                                                                                                                                  bull Define Distance as

                                                                                                                                  x

                                                                                                                                  x

                                                                                                                                  yxyxyxyxd t

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Inner (Dot Scalar) Product

                                                                                                                                  bull for Vectors and

                                                                                                                                  yxyxyx td

                                                                                                                                  jjj

                                                                                                                                  1

                                                                                                                                  x y

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Inner (Dot Scalar) Product

                                                                                                                                  bull for Vectors and

                                                                                                                                  bull Related to Norm via

                                                                                                                                  yxyxyx td

                                                                                                                                  jjj

                                                                                                                                  1

                                                                                                                                  x y

                                                                                                                                  21 xxx

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                                  yyxx

                                                                                                                                  yx

                                                                                                                                  yx

                                                                                                                                  yxyxangle

                                                                                                                                  tt

                                                                                                                                  t

                                                                                                                                  11 cos

                                                                                                                                  cos

                                                                                                                                  x y

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                  if and only if

                                                                                                                                  yyxx

                                                                                                                                  yx

                                                                                                                                  yx

                                                                                                                                  yxyxangle

                                                                                                                                  tt

                                                                                                                                  t

                                                                                                                                  11 cos

                                                                                                                                  cos

                                                                                                                                  x y

                                                                                                                                  yx 0 yx

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis

                                                                                                                                  bull All Orthogonal to each other

                                                                                                                                  ie for

                                                                                                                                  bull All have Length 1

                                                                                                                                  ie for

                                                                                                                                  nvv 1

                                                                                                                                  1 ii vv

                                                                                                                                  0 ii vv ii

                                                                                                                                  ni 1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                  bull Spectral Representation

                                                                                                                                  where

                                                                                                                                  nvv 1

                                                                                                                                  n

                                                                                                                                  iii vax

                                                                                                                                  1

                                                                                                                                  ii vxa

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                  bull Spectral Representation

                                                                                                                                  where

                                                                                                                                  (Coefficient is Inner Product Cool Notation)

                                                                                                                                  nvv 1

                                                                                                                                  n

                                                                                                                                  iii vax

                                                                                                                                  1

                                                                                                                                  ii vxa

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                  bull Spectral Representation

                                                                                                                                  where

                                                                                                                                  Check

                                                                                                                                  nvv 1

                                                                                                                                  n

                                                                                                                                  iii vax

                                                                                                                                  1

                                                                                                                                  ii vxa

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iiii avvavvavx

                                                                                                                                  1

                                                                                                                                  1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                  bull Spectral Representation

                                                                                                                                  where

                                                                                                                                  Check

                                                                                                                                  bull Matrix Notation where ie

                                                                                                                                  For the Basis Matrix

                                                                                                                                  nvv 1

                                                                                                                                  n

                                                                                                                                  iii vax

                                                                                                                                  1

                                                                                                                                  ii vxa

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iiii avvavvavx

                                                                                                                                  1

                                                                                                                                  1

                                                                                                                                  aBx Bxa tt xBa t

                                                                                                                                  nvvB 1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                  bull Spectral Representation

                                                                                                                                  where

                                                                                                                                  Check

                                                                                                                                  bull Matrix Notation where ie

                                                                                                                                  is called transform of

                                                                                                                                  (eg Fourier or Wavelet)

                                                                                                                                  nvv 1

                                                                                                                                  n

                                                                                                                                  iii vax

                                                                                                                                  1

                                                                                                                                  ii vxa

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iii

                                                                                                                                  n

                                                                                                                                  iiii avvavvavx

                                                                                                                                  1

                                                                                                                                  1

                                                                                                                                  aBx Bxa tt xBa t

                                                                                                                                  xa

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Parseval identity for

                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                  x

                                                                                                                                  nvv 1

                                                                                                                                  2

                                                                                                                                  1

                                                                                                                                  22

                                                                                                                                  1

                                                                                                                                  2 aavxx

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Parseval identity for

                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                  bull Pythagorean theorem

                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                  x

                                                                                                                                  nvv 1

                                                                                                                                  2

                                                                                                                                  1

                                                                                                                                  22

                                                                                                                                  1

                                                                                                                                  2 aavxx

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Parseval identity for

                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                  bull Pythagorean theorem

                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                  bull Transform has same length as

                                                                                                                                  ie ldquorotation in rdquo

                                                                                                                                  x

                                                                                                                                  nvv 1

                                                                                                                                  2

                                                                                                                                  1

                                                                                                                                  22

                                                                                                                                  1

                                                                                                                                  2 aavxx

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  a xd

                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                  xV

                                                                                                                                  V

                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                  bull Find that Solves

                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                  xV

                                                                                                                                  V

                                                                                                                                  VxPV vxVv

                                                                                                                                  min

                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                  bull Find that Solves

                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                  bull For Inner Product (Hilbert) Space

                                                                                                                                  Exists and is Unique

                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                  xV

                                                                                                                                  V

                                                                                                                                  VxPV vxVv

                                                                                                                                  min

                                                                                                                                  xPV

                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  d VB

                                                                                                                                  xBBBBxP tVV

                                                                                                                                  tVVV

                                                                                                                                  1

                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  d VB

                                                                                                                                  xBBBBxP tVV

                                                                                                                                  tVVV

                                                                                                                                  1

                                                                                                                                  tVV

                                                                                                                                  tVVV BBBBP

                                                                                                                                  1

                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                  (note same operation underlies least squares)

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  d VB

                                                                                                                                  xBBBBxP tVV

                                                                                                                                  tVVV

                                                                                                                                  1

                                                                                                                                  tVV

                                                                                                                                  tVVV BBBBP

                                                                                                                                  1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                  nnVtV IBB

                                                                                                                                  10

                                                                                                                                  01

                                                                                                                                  1

                                                                                                                                  111

                                                                                                                                  1

                                                                                                                                  1

                                                                                                                                  nnn

                                                                                                                                  n

                                                                                                                                  ntn

                                                                                                                                  t

                                                                                                                                  vvvv

                                                                                                                                  vvvv

                                                                                                                                  vv

                                                                                                                                  v

                                                                                                                                  v

                                                                                                                                  nvv 1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                  bull So =

                                                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                  (Recall Right Multrsquon)

                                                                                                                                  nnVtV IBB

                                                                                                                                  10

                                                                                                                                  01

                                                                                                                                  1

                                                                                                                                  111

                                                                                                                                  1

                                                                                                                                  1

                                                                                                                                  nnn

                                                                                                                                  n

                                                                                                                                  ntn

                                                                                                                                  t

                                                                                                                                  vvvv

                                                                                                                                  vvvv

                                                                                                                                  vv

                                                                                                                                  v

                                                                                                                                  v

                                                                                                                                  xBBxP tVVV

                                                                                                                                  x V

                                                                                                                                  nvv 1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                  and

                                                                                                                                  V

                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                  and

                                                                                                                                  bull Parseval Inequality

                                                                                                                                  V

                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                  2

                                                                                                                                  1

                                                                                                                                  22

                                                                                                                                  1

                                                                                                                                  22 aavxxxP

                                                                                                                                  n

                                                                                                                                  ii

                                                                                                                                  n

                                                                                                                                  iiV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                  (So All of Above Applies)

                                                                                                                                  ddU IUU t

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                  (So All of Above Applies)

                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                  ddU IUU t

                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                  i ii 2

                                                                                                                                  1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                  (So All of Above Applies)

                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                  ddU IUU t

                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                  i ii 2

                                                                                                                                  1

                                                                                                                                  U

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                  For a Matrix

                                                                                                                                  Find

                                                                                                                                  ndX

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                  For a Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  with Entries

                                                                                                                                  called Singular Values

                                                                                                                                  ndX

                                                                                                                                  ndS

                                                                                                                                  )min(1 ndss

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                  For a Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  with Entries

                                                                                                                                  called Singular Values

                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                  (recall )

                                                                                                                                  ndX

                                                                                                                                  ndS

                                                                                                                                  )min(1 ndss

                                                                                                                                  ddU nnV

                                                                                                                                  IVVUU tt

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                  For a Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  with Entries

                                                                                                                                  called Singular Values

                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                  (recall )

                                                                                                                                  So That

                                                                                                                                  ndX

                                                                                                                                  ndS

                                                                                                                                  )min(1 ndss

                                                                                                                                  ddU nnV

                                                                                                                                  IVVUU tt tUSVX

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                  bull First rotate

                                                                                                                                  X

                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                  bull First rotate

                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                  X

                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                  is

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                  bull First rotate

                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                  bull Third rotate again

                                                                                                                                  X

                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                  is

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                  bull First rotate

                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                  bull Third rotate again

                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                  X

                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                  is

                                                                                                                                  SVD Compact Representation

                                                                                                                                  Useful Labeling

                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  )min(1 dnss

                                                                                                                                  SVD Compact Representation

                                                                                                                                  Useful Labeling

                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  )min(1 dnss

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  r

                                                                                                                                  SVD Compact Representation

                                                                                                                                  Useful Labeling

                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                  Let = of positive singular values

                                                                                                                                  )min(1 dnss

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  r

                                                                                                                                  SVD Compact Representation

                                                                                                                                  Useful Labeling

                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                  Let = of positive singular values

                                                                                                                                  Then

                                                                                                                                  Where are truncations of

                                                                                                                                  trnrrrd VSUX

                                                                                                                                  VSU

                                                                                                                                  )min(1 dnss

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Full Representation

                                                                                                                                  =

                                                                                                                                  Graphics Display Assumes

                                                                                                                                  ndX ddU ndS nn

                                                                                                                                  tV

                                                                                                                                  nd

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Full Representation

                                                                                                                                  =

                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                  ndX ddU ndS nn

                                                                                                                                  tV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Full Representation

                                                                                                                                  =

                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                  All 0s in Bottom

                                                                                                                                  ndX ddU ndS nn

                                                                                                                                  tV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Reduced Representation

                                                                                                                                  =

                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                  ndX ddU nnS nn

                                                                                                                                  tV

                                                                                                                                  nnd 0

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Reduced Representation

                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                  tV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Reduced Representation

                                                                                                                                  =

                                                                                                                                  Also Some of These May be 0

                                                                                                                                  ndX ndU nnS nn

                                                                                                                                  tV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Compact Representation

                                                                                                                                  = ndX rdU

                                                                                                                                  rrS nrtV

                                                                                                                                  0

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Compact Representation

                                                                                                                                  =

                                                                                                                                  These Get 0ed Out

                                                                                                                                  ndX rdU

                                                                                                                                  rrS nrtV

                                                                                                                                  0

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  SVD Compact Representation

                                                                                                                                  = ndX rdU

                                                                                                                                  rrS nrtV

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                  Find

                                                                                                                                  ddX

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  Called Eigenvalues

                                                                                                                                  Convenient Ordering

                                                                                                                                  ddX

                                                                                                                                  d

                                                                                                                                  D

                                                                                                                                  0

                                                                                                                                  01

                                                                                                                                  n 1

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                  (ie )

                                                                                                                                  ddX

                                                                                                                                  d

                                                                                                                                  D

                                                                                                                                  0

                                                                                                                                  01

                                                                                                                                  ddB

                                                                                                                                  ddtt IBBBB

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                  (ie )

                                                                                                                                  So that ie

                                                                                                                                  ddX

                                                                                                                                  d

                                                                                                                                  D

                                                                                                                                  0

                                                                                                                                  01

                                                                                                                                  ddB

                                                                                                                                  ddtt IBBBB

                                                                                                                                  DBBX tBDBX

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                  (looks similar)

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                  Complex (uses )

                                                                                                                                  VU

                                                                                                                                  1i

                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                  VU

                                                                                                                                  X

                                                                                                                                  BVU

                                                                                                                                  1i

                                                                                                                                  • Participant Presentations
                                                                                                                                  • PCA to find clusters
                                                                                                                                  • Statistical Smoothing
                                                                                                                                  • Kernel Density Estimation
                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                  • SiZer Background
                                                                                                                                  • SiZer Background (2)
                                                                                                                                  • SiZer Background (3)
                                                                                                                                  • SiZer Background (4)
                                                                                                                                  • SiZer Background (5)
                                                                                                                                  • SiZer Overview
                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                  • New Subpopulation View
                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                  • Detailed Look at PCA
                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                  • An Interesting Historical Note
                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                  • Review of Linear Algebra
                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Norm of a Vector

                                                                                                                                    bull in d 21

                                                                                                                                    21

                                                                                                                                    1

                                                                                                                                    2 xxxx td

                                                                                                                                    jj

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Norm of a Vector

                                                                                                                                    bull in

                                                                                                                                    bull Idea length of the vector

                                                                                                                                    d 2121

                                                                                                                                    1

                                                                                                                                    2 xxxx td

                                                                                                                                    jj

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Norm of a Vector

                                                                                                                                    bull in

                                                                                                                                    bull Idea length of the vector

                                                                                                                                    bull Note strange properties for high

                                                                                                                                    eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                                    d 2121

                                                                                                                                    1

                                                                                                                                    2 xxxx td

                                                                                                                                    jj

                                                                                                                                    d

                                                                                                                                    d

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Norm of a Vector (cont)

                                                                                                                                    bull Length Normalized Vector

                                                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                    amp is a Direction Vector)

                                                                                                                                    x

                                                                                                                                    x

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Norm of a Vector (cont)

                                                                                                                                    bull Length Normalized Vector

                                                                                                                                    (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                    amp is a Direction Vector)

                                                                                                                                    bull Define Distance as

                                                                                                                                    x

                                                                                                                                    x

                                                                                                                                    yxyxyxyxd t

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Inner (Dot Scalar) Product

                                                                                                                                    bull for Vectors and

                                                                                                                                    yxyxyx td

                                                                                                                                    jjj

                                                                                                                                    1

                                                                                                                                    x y

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Inner (Dot Scalar) Product

                                                                                                                                    bull for Vectors and

                                                                                                                                    bull Related to Norm via

                                                                                                                                    yxyxyx td

                                                                                                                                    jjj

                                                                                                                                    1

                                                                                                                                    x y

                                                                                                                                    21 xxx

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                                                    yyxx

                                                                                                                                    yx

                                                                                                                                    yx

                                                                                                                                    yxyxangle

                                                                                                                                    tt

                                                                                                                                    t

                                                                                                                                    11 cos

                                                                                                                                    cos

                                                                                                                                    x y

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                    if and only if

                                                                                                                                    yyxx

                                                                                                                                    yx

                                                                                                                                    yx

                                                                                                                                    yxyxangle

                                                                                                                                    tt

                                                                                                                                    t

                                                                                                                                    11 cos

                                                                                                                                    cos

                                                                                                                                    x y

                                                                                                                                    yx 0 yx

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis

                                                                                                                                    bull All Orthogonal to each other

                                                                                                                                    ie for

                                                                                                                                    bull All have Length 1

                                                                                                                                    ie for

                                                                                                                                    nvv 1

                                                                                                                                    1 ii vv

                                                                                                                                    0 ii vv ii

                                                                                                                                    ni 1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                    bull Spectral Representation

                                                                                                                                    where

                                                                                                                                    nvv 1

                                                                                                                                    n

                                                                                                                                    iii vax

                                                                                                                                    1

                                                                                                                                    ii vxa

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                    bull Spectral Representation

                                                                                                                                    where

                                                                                                                                    (Coefficient is Inner Product Cool Notation)

                                                                                                                                    nvv 1

                                                                                                                                    n

                                                                                                                                    iii vax

                                                                                                                                    1

                                                                                                                                    ii vxa

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                    bull Spectral Representation

                                                                                                                                    where

                                                                                                                                    Check

                                                                                                                                    nvv 1

                                                                                                                                    n

                                                                                                                                    iii vax

                                                                                                                                    1

                                                                                                                                    ii vxa

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iiii avvavvavx

                                                                                                                                    1

                                                                                                                                    1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                    bull Spectral Representation

                                                                                                                                    where

                                                                                                                                    Check

                                                                                                                                    bull Matrix Notation where ie

                                                                                                                                    For the Basis Matrix

                                                                                                                                    nvv 1

                                                                                                                                    n

                                                                                                                                    iii vax

                                                                                                                                    1

                                                                                                                                    ii vxa

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iiii avvavvavx

                                                                                                                                    1

                                                                                                                                    1

                                                                                                                                    aBx Bxa tt xBa t

                                                                                                                                    nvvB 1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                    bull Spectral Representation

                                                                                                                                    where

                                                                                                                                    Check

                                                                                                                                    bull Matrix Notation where ie

                                                                                                                                    is called transform of

                                                                                                                                    (eg Fourier or Wavelet)

                                                                                                                                    nvv 1

                                                                                                                                    n

                                                                                                                                    iii vax

                                                                                                                                    1

                                                                                                                                    ii vxa

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iii

                                                                                                                                    n

                                                                                                                                    iiii avvavvavx

                                                                                                                                    1

                                                                                                                                    1

                                                                                                                                    aBx Bxa tt xBa t

                                                                                                                                    xa

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Parseval identity for

                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                    x

                                                                                                                                    nvv 1

                                                                                                                                    2

                                                                                                                                    1

                                                                                                                                    22

                                                                                                                                    1

                                                                                                                                    2 aavxx

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Parseval identity for

                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                    bull Pythagorean theorem

                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                    x

                                                                                                                                    nvv 1

                                                                                                                                    2

                                                                                                                                    1

                                                                                                                                    22

                                                                                                                                    1

                                                                                                                                    2 aavxx

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Parseval identity for

                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                    bull Pythagorean theorem

                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                    bull Transform has same length as

                                                                                                                                    ie ldquorotation in rdquo

                                                                                                                                    x

                                                                                                                                    nvv 1

                                                                                                                                    2

                                                                                                                                    1

                                                                                                                                    22

                                                                                                                                    1

                                                                                                                                    2 aavxx

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    a xd

                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                    xV

                                                                                                                                    V

                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                    bull Find that Solves

                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                    xV

                                                                                                                                    V

                                                                                                                                    VxPV vxVv

                                                                                                                                    min

                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                    bull Find that Solves

                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                    bull For Inner Product (Hilbert) Space

                                                                                                                                    Exists and is Unique

                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                    xV

                                                                                                                                    V

                                                                                                                                    VxPV vxVv

                                                                                                                                    min

                                                                                                                                    xPV

                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    d VB

                                                                                                                                    xBBBBxP tVV

                                                                                                                                    tVVV

                                                                                                                                    1

                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    d VB

                                                                                                                                    xBBBBxP tVV

                                                                                                                                    tVVV

                                                                                                                                    1

                                                                                                                                    tVV

                                                                                                                                    tVVV BBBBP

                                                                                                                                    1

                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                    (note same operation underlies least squares)

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    d VB

                                                                                                                                    xBBBBxP tVV

                                                                                                                                    tVVV

                                                                                                                                    1

                                                                                                                                    tVV

                                                                                                                                    tVVV BBBBP

                                                                                                                                    1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                    nnVtV IBB

                                                                                                                                    10

                                                                                                                                    01

                                                                                                                                    1

                                                                                                                                    111

                                                                                                                                    1

                                                                                                                                    1

                                                                                                                                    nnn

                                                                                                                                    n

                                                                                                                                    ntn

                                                                                                                                    t

                                                                                                                                    vvvv

                                                                                                                                    vvvv

                                                                                                                                    vv

                                                                                                                                    v

                                                                                                                                    v

                                                                                                                                    nvv 1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                    bull So =

                                                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                    (Recall Right Multrsquon)

                                                                                                                                    nnVtV IBB

                                                                                                                                    10

                                                                                                                                    01

                                                                                                                                    1

                                                                                                                                    111

                                                                                                                                    1

                                                                                                                                    1

                                                                                                                                    nnn

                                                                                                                                    n

                                                                                                                                    ntn

                                                                                                                                    t

                                                                                                                                    vvvv

                                                                                                                                    vvvv

                                                                                                                                    vv

                                                                                                                                    v

                                                                                                                                    v

                                                                                                                                    xBBxP tVVV

                                                                                                                                    x V

                                                                                                                                    nvv 1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                    and

                                                                                                                                    V

                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                    and

                                                                                                                                    bull Parseval Inequality

                                                                                                                                    V

                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                    2

                                                                                                                                    1

                                                                                                                                    22

                                                                                                                                    1

                                                                                                                                    22 aavxxxP

                                                                                                                                    n

                                                                                                                                    ii

                                                                                                                                    n

                                                                                                                                    iiV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                    (So All of Above Applies)

                                                                                                                                    ddU IUU t

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                    (So All of Above Applies)

                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                    ddU IUU t

                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                    i ii 2

                                                                                                                                    1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                    (So All of Above Applies)

                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                    ddU IUU t

                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                    i ii 2

                                                                                                                                    1

                                                                                                                                    U

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                    For a Matrix

                                                                                                                                    Find

                                                                                                                                    ndX

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                    For a Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    with Entries

                                                                                                                                    called Singular Values

                                                                                                                                    ndX

                                                                                                                                    ndS

                                                                                                                                    )min(1 ndss

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                    For a Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    with Entries

                                                                                                                                    called Singular Values

                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                    (recall )

                                                                                                                                    ndX

                                                                                                                                    ndS

                                                                                                                                    )min(1 ndss

                                                                                                                                    ddU nnV

                                                                                                                                    IVVUU tt

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                    For a Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    with Entries

                                                                                                                                    called Singular Values

                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                    (recall )

                                                                                                                                    So That

                                                                                                                                    ndX

                                                                                                                                    ndS

                                                                                                                                    )min(1 ndss

                                                                                                                                    ddU nnV

                                                                                                                                    IVVUU tt tUSVX

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                    bull First rotate

                                                                                                                                    X

                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                    bull First rotate

                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                    X

                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                    is

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                    bull First rotate

                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                    bull Third rotate again

                                                                                                                                    X

                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                    is

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                    bull First rotate

                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                    bull Third rotate again

                                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                                    X

                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                    is

                                                                                                                                    SVD Compact Representation

                                                                                                                                    Useful Labeling

                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    )min(1 dnss

                                                                                                                                    SVD Compact Representation

                                                                                                                                    Useful Labeling

                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    )min(1 dnss

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    r

                                                                                                                                    SVD Compact Representation

                                                                                                                                    Useful Labeling

                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                    Let = of positive singular values

                                                                                                                                    )min(1 dnss

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    r

                                                                                                                                    SVD Compact Representation

                                                                                                                                    Useful Labeling

                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                    Let = of positive singular values

                                                                                                                                    Then

                                                                                                                                    Where are truncations of

                                                                                                                                    trnrrrd VSUX

                                                                                                                                    VSU

                                                                                                                                    )min(1 dnss

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Full Representation

                                                                                                                                    =

                                                                                                                                    Graphics Display Assumes

                                                                                                                                    ndX ddU ndS nn

                                                                                                                                    tV

                                                                                                                                    nd

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Full Representation

                                                                                                                                    =

                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                    ndX ddU ndS nn

                                                                                                                                    tV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Full Representation

                                                                                                                                    =

                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                    All 0s in Bottom

                                                                                                                                    ndX ddU ndS nn

                                                                                                                                    tV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Reduced Representation

                                                                                                                                    =

                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                    ndX ddU nnS nn

                                                                                                                                    tV

                                                                                                                                    nnd 0

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Reduced Representation

                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                    tV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Reduced Representation

                                                                                                                                    =

                                                                                                                                    Also Some of These May be 0

                                                                                                                                    ndX ndU nnS nn

                                                                                                                                    tV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Compact Representation

                                                                                                                                    = ndX rdU

                                                                                                                                    rrS nrtV

                                                                                                                                    0

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Compact Representation

                                                                                                                                    =

                                                                                                                                    These Get 0ed Out

                                                                                                                                    ndX rdU

                                                                                                                                    rrS nrtV

                                                                                                                                    0

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    SVD Compact Representation

                                                                                                                                    = ndX rdU

                                                                                                                                    rrS nrtV

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                    Find

                                                                                                                                    ddX

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    Called Eigenvalues

                                                                                                                                    Convenient Ordering

                                                                                                                                    ddX

                                                                                                                                    d

                                                                                                                                    D

                                                                                                                                    0

                                                                                                                                    01

                                                                                                                                    n 1

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                    (ie )

                                                                                                                                    ddX

                                                                                                                                    d

                                                                                                                                    D

                                                                                                                                    0

                                                                                                                                    01

                                                                                                                                    ddB

                                                                                                                                    ddtt IBBBB

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                    (ie )

                                                                                                                                    So that ie

                                                                                                                                    ddX

                                                                                                                                    d

                                                                                                                                    D

                                                                                                                                    0

                                                                                                                                    01

                                                                                                                                    ddB

                                                                                                                                    ddtt IBBBB

                                                                                                                                    DBBX tBDBX

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                    (looks similar)

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                    Complex (uses )

                                                                                                                                    VU

                                                                                                                                    1i

                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                    VU

                                                                                                                                    X

                                                                                                                                    BVU

                                                                                                                                    1i

                                                                                                                                    • Participant Presentations
                                                                                                                                    • PCA to find clusters
                                                                                                                                    • Statistical Smoothing
                                                                                                                                    • Kernel Density Estimation
                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                    • SiZer Background
                                                                                                                                    • SiZer Background (2)
                                                                                                                                    • SiZer Background (3)
                                                                                                                                    • SiZer Background (4)
                                                                                                                                    • SiZer Background (5)
                                                                                                                                    • SiZer Overview
                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                    • New Subpopulation View
                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                    • Detailed Look at PCA
                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                    • An Interesting Historical Note
                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                    • Review of Linear Algebra
                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Norm of a Vector

                                                                                                                                      bull in

                                                                                                                                      bull Idea length of the vector

                                                                                                                                      d 2121

                                                                                                                                      1

                                                                                                                                      2 xxxx td

                                                                                                                                      jj

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Norm of a Vector

                                                                                                                                      bull in

                                                                                                                                      bull Idea length of the vector

                                                                                                                                      bull Note strange properties for high

                                                                                                                                      eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                                      d 2121

                                                                                                                                      1

                                                                                                                                      2 xxxx td

                                                                                                                                      jj

                                                                                                                                      d

                                                                                                                                      d

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Norm of a Vector (cont)

                                                                                                                                      bull Length Normalized Vector

                                                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                      amp is a Direction Vector)

                                                                                                                                      x

                                                                                                                                      x

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Norm of a Vector (cont)

                                                                                                                                      bull Length Normalized Vector

                                                                                                                                      (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                      amp is a Direction Vector)

                                                                                                                                      bull Define Distance as

                                                                                                                                      x

                                                                                                                                      x

                                                                                                                                      yxyxyxyxd t

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Inner (Dot Scalar) Product

                                                                                                                                      bull for Vectors and

                                                                                                                                      yxyxyx td

                                                                                                                                      jjj

                                                                                                                                      1

                                                                                                                                      x y

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Inner (Dot Scalar) Product

                                                                                                                                      bull for Vectors and

                                                                                                                                      bull Related to Norm via

                                                                                                                                      yxyxyx td

                                                                                                                                      jjj

                                                                                                                                      1

                                                                                                                                      x y

                                                                                                                                      21 xxx

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                                                      yyxx

                                                                                                                                      yx

                                                                                                                                      yx

                                                                                                                                      yxyxangle

                                                                                                                                      tt

                                                                                                                                      t

                                                                                                                                      11 cos

                                                                                                                                      cos

                                                                                                                                      x y

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Inner (Dot Scalar) Product (cont)

                                                                                                                                      bull measures ldquoangle between and rdquo as

                                                                                                                                      bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                      if and only if

                                                                                                                                      yyxx

                                                                                                                                      yx

                                                                                                                                      yx

                                                                                                                                      yxyxangle

                                                                                                                                      tt

                                                                                                                                      t

                                                                                                                                      11 cos

                                                                                                                                      cos

                                                                                                                                      x y

                                                                                                                                      yx 0 yx

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis

                                                                                                                                      bull All Orthogonal to each other

                                                                                                                                      ie for

                                                                                                                                      bull All have Length 1

                                                                                                                                      ie for

                                                                                                                                      nvv 1

                                                                                                                                      1 ii vv

                                                                                                                                      0 ii vv ii

                                                                                                                                      ni 1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                      bull Spectral Representation

                                                                                                                                      where

                                                                                                                                      nvv 1

                                                                                                                                      n

                                                                                                                                      iii vax

                                                                                                                                      1

                                                                                                                                      ii vxa

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                      bull Spectral Representation

                                                                                                                                      where

                                                                                                                                      (Coefficient is Inner Product Cool Notation)

                                                                                                                                      nvv 1

                                                                                                                                      n

                                                                                                                                      iii vax

                                                                                                                                      1

                                                                                                                                      ii vxa

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                      bull Spectral Representation

                                                                                                                                      where

                                                                                                                                      Check

                                                                                                                                      nvv 1

                                                                                                                                      n

                                                                                                                                      iii vax

                                                                                                                                      1

                                                                                                                                      ii vxa

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iiii avvavvavx

                                                                                                                                      1

                                                                                                                                      1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                      bull Spectral Representation

                                                                                                                                      where

                                                                                                                                      Check

                                                                                                                                      bull Matrix Notation where ie

                                                                                                                                      For the Basis Matrix

                                                                                                                                      nvv 1

                                                                                                                                      n

                                                                                                                                      iii vax

                                                                                                                                      1

                                                                                                                                      ii vxa

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iiii avvavvavx

                                                                                                                                      1

                                                                                                                                      1

                                                                                                                                      aBx Bxa tt xBa t

                                                                                                                                      nvvB 1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                      bull Spectral Representation

                                                                                                                                      where

                                                                                                                                      Check

                                                                                                                                      bull Matrix Notation where ie

                                                                                                                                      is called transform of

                                                                                                                                      (eg Fourier or Wavelet)

                                                                                                                                      nvv 1

                                                                                                                                      n

                                                                                                                                      iii vax

                                                                                                                                      1

                                                                                                                                      ii vxa

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iii

                                                                                                                                      n

                                                                                                                                      iiii avvavvavx

                                                                                                                                      1

                                                                                                                                      1

                                                                                                                                      aBx Bxa tt xBa t

                                                                                                                                      xa

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Parseval identity for

                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                      x

                                                                                                                                      nvv 1

                                                                                                                                      2

                                                                                                                                      1

                                                                                                                                      22

                                                                                                                                      1

                                                                                                                                      2 aavxx

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Parseval identity for

                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                      bull Pythagorean theorem

                                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                                      bull ANOVA - sums of squares

                                                                                                                                      x

                                                                                                                                      nvv 1

                                                                                                                                      2

                                                                                                                                      1

                                                                                                                                      22

                                                                                                                                      1

                                                                                                                                      2 aavxx

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Parseval identity for

                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                      bull Pythagorean theorem

                                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                                      bull ANOVA - sums of squares

                                                                                                                                      bull Transform has same length as

                                                                                                                                      ie ldquorotation in rdquo

                                                                                                                                      x

                                                                                                                                      nvv 1

                                                                                                                                      2

                                                                                                                                      1

                                                                                                                                      22

                                                                                                                                      1

                                                                                                                                      2 aavxx

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      a xd

                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                      xV

                                                                                                                                      V

                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                      bull Find that Solves

                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                      xV

                                                                                                                                      V

                                                                                                                                      VxPV vxVv

                                                                                                                                      min

                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                      bull Find that Solves

                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                      bull For Inner Product (Hilbert) Space

                                                                                                                                      Exists and is Unique

                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                      xV

                                                                                                                                      V

                                                                                                                                      VxPV vxVv

                                                                                                                                      min

                                                                                                                                      xPV

                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      d VB

                                                                                                                                      xBBBBxP tVV

                                                                                                                                      tVVV

                                                                                                                                      1

                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      d VB

                                                                                                                                      xBBBBxP tVV

                                                                                                                                      tVVV

                                                                                                                                      1

                                                                                                                                      tVV

                                                                                                                                      tVVV BBBBP

                                                                                                                                      1

                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                      (note same operation underlies least squares)

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      d VB

                                                                                                                                      xBBBBxP tVV

                                                                                                                                      tVVV

                                                                                                                                      1

                                                                                                                                      tVV

                                                                                                                                      tVVV BBBBP

                                                                                                                                      1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                      nnVtV IBB

                                                                                                                                      10

                                                                                                                                      01

                                                                                                                                      1

                                                                                                                                      111

                                                                                                                                      1

                                                                                                                                      1

                                                                                                                                      nnn

                                                                                                                                      n

                                                                                                                                      ntn

                                                                                                                                      t

                                                                                                                                      vvvv

                                                                                                                                      vvvv

                                                                                                                                      vv

                                                                                                                                      v

                                                                                                                                      v

                                                                                                                                      nvv 1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                      bull So =

                                                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                      (Recall Right Multrsquon)

                                                                                                                                      nnVtV IBB

                                                                                                                                      10

                                                                                                                                      01

                                                                                                                                      1

                                                                                                                                      111

                                                                                                                                      1

                                                                                                                                      1

                                                                                                                                      nnn

                                                                                                                                      n

                                                                                                                                      ntn

                                                                                                                                      t

                                                                                                                                      vvvv

                                                                                                                                      vvvv

                                                                                                                                      vv

                                                                                                                                      v

                                                                                                                                      v

                                                                                                                                      xBBxP tVVV

                                                                                                                                      x V

                                                                                                                                      nvv 1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                      and

                                                                                                                                      V

                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                      and

                                                                                                                                      bull Parseval Inequality

                                                                                                                                      V

                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                      2

                                                                                                                                      1

                                                                                                                                      22

                                                                                                                                      1

                                                                                                                                      22 aavxxxP

                                                                                                                                      n

                                                                                                                                      ii

                                                                                                                                      n

                                                                                                                                      iiV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                      (So All of Above Applies)

                                                                                                                                      ddU IUU t

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                      (So All of Above Applies)

                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                      ddU IUU t

                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                      i ii 2

                                                                                                                                      1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                      (So All of Above Applies)

                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                      ddU IUU t

                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                      i ii 2

                                                                                                                                      1

                                                                                                                                      U

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                      For a Matrix

                                                                                                                                      Find

                                                                                                                                      ndX

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                      For a Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      with Entries

                                                                                                                                      called Singular Values

                                                                                                                                      ndX

                                                                                                                                      ndS

                                                                                                                                      )min(1 ndss

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                      For a Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      with Entries

                                                                                                                                      called Singular Values

                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                      (recall )

                                                                                                                                      ndX

                                                                                                                                      ndS

                                                                                                                                      )min(1 ndss

                                                                                                                                      ddU nnV

                                                                                                                                      IVVUU tt

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                      For a Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      with Entries

                                                                                                                                      called Singular Values

                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                      (recall )

                                                                                                                                      So That

                                                                                                                                      ndX

                                                                                                                                      ndS

                                                                                                                                      )min(1 ndss

                                                                                                                                      ddU nnV

                                                                                                                                      IVVUU tt tUSVX

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                      bull First rotate

                                                                                                                                      X

                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                      bull First rotate

                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                      X

                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                      is

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                      bull First rotate

                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                      bull Third rotate again

                                                                                                                                      X

                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                      is

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                      bull First rotate

                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                      bull Third rotate again

                                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                                      X

                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                      is

                                                                                                                                      SVD Compact Representation

                                                                                                                                      Useful Labeling

                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      )min(1 dnss

                                                                                                                                      SVD Compact Representation

                                                                                                                                      Useful Labeling

                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      )min(1 dnss

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      r

                                                                                                                                      SVD Compact Representation

                                                                                                                                      Useful Labeling

                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                      Let = of positive singular values

                                                                                                                                      )min(1 dnss

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      r

                                                                                                                                      SVD Compact Representation

                                                                                                                                      Useful Labeling

                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                      Let = of positive singular values

                                                                                                                                      Then

                                                                                                                                      Where are truncations of

                                                                                                                                      trnrrrd VSUX

                                                                                                                                      VSU

                                                                                                                                      )min(1 dnss

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Full Representation

                                                                                                                                      =

                                                                                                                                      Graphics Display Assumes

                                                                                                                                      ndX ddU ndS nn

                                                                                                                                      tV

                                                                                                                                      nd

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Full Representation

                                                                                                                                      =

                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                      ndX ddU ndS nn

                                                                                                                                      tV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Full Representation

                                                                                                                                      =

                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                      All 0s in Bottom

                                                                                                                                      ndX ddU ndS nn

                                                                                                                                      tV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Reduced Representation

                                                                                                                                      =

                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                      ndX ddU nnS nn

                                                                                                                                      tV

                                                                                                                                      nnd 0

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Reduced Representation

                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                      tV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Reduced Representation

                                                                                                                                      =

                                                                                                                                      Also Some of These May be 0

                                                                                                                                      ndX ndU nnS nn

                                                                                                                                      tV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Compact Representation

                                                                                                                                      = ndX rdU

                                                                                                                                      rrS nrtV

                                                                                                                                      0

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Compact Representation

                                                                                                                                      =

                                                                                                                                      These Get 0ed Out

                                                                                                                                      ndX rdU

                                                                                                                                      rrS nrtV

                                                                                                                                      0

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      SVD Compact Representation

                                                                                                                                      = ndX rdU

                                                                                                                                      rrS nrtV

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                      Find

                                                                                                                                      ddX

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      Called Eigenvalues

                                                                                                                                      Convenient Ordering

                                                                                                                                      ddX

                                                                                                                                      d

                                                                                                                                      D

                                                                                                                                      0

                                                                                                                                      01

                                                                                                                                      n 1

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                      (ie )

                                                                                                                                      ddX

                                                                                                                                      d

                                                                                                                                      D

                                                                                                                                      0

                                                                                                                                      01

                                                                                                                                      ddB

                                                                                                                                      ddtt IBBBB

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                      (ie )

                                                                                                                                      So that ie

                                                                                                                                      ddX

                                                                                                                                      d

                                                                                                                                      D

                                                                                                                                      0

                                                                                                                                      01

                                                                                                                                      ddB

                                                                                                                                      ddtt IBBBB

                                                                                                                                      DBBX tBDBX

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                      (looks similar)

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                      Complex (uses )

                                                                                                                                      VU

                                                                                                                                      1i

                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                      VU

                                                                                                                                      X

                                                                                                                                      BVU

                                                                                                                                      1i

                                                                                                                                      • Participant Presentations
                                                                                                                                      • PCA to find clusters
                                                                                                                                      • Statistical Smoothing
                                                                                                                                      • Kernel Density Estimation
                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                      • SiZer Background
                                                                                                                                      • SiZer Background (2)
                                                                                                                                      • SiZer Background (3)
                                                                                                                                      • SiZer Background (4)
                                                                                                                                      • SiZer Background (5)
                                                                                                                                      • SiZer Overview
                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                      • New Subpopulation View
                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                      • Detailed Look at PCA
                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                      • An Interesting Historical Note
                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                      • Review of Linear Algebra
                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Norm of a Vector

                                                                                                                                        bull in

                                                                                                                                        bull Idea length of the vector

                                                                                                                                        bull Note strange properties for high

                                                                                                                                        eg ldquolength of diagonal of unit cuberdquo =

                                                                                                                                        d 2121

                                                                                                                                        1

                                                                                                                                        2 xxxx td

                                                                                                                                        jj

                                                                                                                                        d

                                                                                                                                        d

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Norm of a Vector (cont)

                                                                                                                                        bull Length Normalized Vector

                                                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                        amp is a Direction Vector)

                                                                                                                                        x

                                                                                                                                        x

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Norm of a Vector (cont)

                                                                                                                                        bull Length Normalized Vector

                                                                                                                                        (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                        amp is a Direction Vector)

                                                                                                                                        bull Define Distance as

                                                                                                                                        x

                                                                                                                                        x

                                                                                                                                        yxyxyxyxd t

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Inner (Dot Scalar) Product

                                                                                                                                        bull for Vectors and

                                                                                                                                        yxyxyx td

                                                                                                                                        jjj

                                                                                                                                        1

                                                                                                                                        x y

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Inner (Dot Scalar) Product

                                                                                                                                        bull for Vectors and

                                                                                                                                        bull Related to Norm via

                                                                                                                                        yxyxyx td

                                                                                                                                        jjj

                                                                                                                                        1

                                                                                                                                        x y

                                                                                                                                        21 xxx

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                                                        yyxx

                                                                                                                                        yx

                                                                                                                                        yx

                                                                                                                                        yxyxangle

                                                                                                                                        tt

                                                                                                                                        t

                                                                                                                                        11 cos

                                                                                                                                        cos

                                                                                                                                        x y

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Inner (Dot Scalar) Product (cont)

                                                                                                                                        bull measures ldquoangle between and rdquo as

                                                                                                                                        bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                        if and only if

                                                                                                                                        yyxx

                                                                                                                                        yx

                                                                                                                                        yx

                                                                                                                                        yxyxangle

                                                                                                                                        tt

                                                                                                                                        t

                                                                                                                                        11 cos

                                                                                                                                        cos

                                                                                                                                        x y

                                                                                                                                        yx 0 yx

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis

                                                                                                                                        bull All Orthogonal to each other

                                                                                                                                        ie for

                                                                                                                                        bull All have Length 1

                                                                                                                                        ie for

                                                                                                                                        nvv 1

                                                                                                                                        1 ii vv

                                                                                                                                        0 ii vv ii

                                                                                                                                        ni 1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                        bull Spectral Representation

                                                                                                                                        where

                                                                                                                                        nvv 1

                                                                                                                                        n

                                                                                                                                        iii vax

                                                                                                                                        1

                                                                                                                                        ii vxa

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                        bull Spectral Representation

                                                                                                                                        where

                                                                                                                                        (Coefficient is Inner Product Cool Notation)

                                                                                                                                        nvv 1

                                                                                                                                        n

                                                                                                                                        iii vax

                                                                                                                                        1

                                                                                                                                        ii vxa

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                        bull Spectral Representation

                                                                                                                                        where

                                                                                                                                        Check

                                                                                                                                        nvv 1

                                                                                                                                        n

                                                                                                                                        iii vax

                                                                                                                                        1

                                                                                                                                        ii vxa

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iiii avvavvavx

                                                                                                                                        1

                                                                                                                                        1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                        bull Spectral Representation

                                                                                                                                        where

                                                                                                                                        Check

                                                                                                                                        bull Matrix Notation where ie

                                                                                                                                        For the Basis Matrix

                                                                                                                                        nvv 1

                                                                                                                                        n

                                                                                                                                        iii vax

                                                                                                                                        1

                                                                                                                                        ii vxa

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iiii avvavvavx

                                                                                                                                        1

                                                                                                                                        1

                                                                                                                                        aBx Bxa tt xBa t

                                                                                                                                        nvvB 1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                        bull Spectral Representation

                                                                                                                                        where

                                                                                                                                        Check

                                                                                                                                        bull Matrix Notation where ie

                                                                                                                                        is called transform of

                                                                                                                                        (eg Fourier or Wavelet)

                                                                                                                                        nvv 1

                                                                                                                                        n

                                                                                                                                        iii vax

                                                                                                                                        1

                                                                                                                                        ii vxa

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iii

                                                                                                                                        n

                                                                                                                                        iiii avvavvavx

                                                                                                                                        1

                                                                                                                                        1

                                                                                                                                        aBx Bxa tt xBa t

                                                                                                                                        xa

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Parseval identity for

                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                        x

                                                                                                                                        nvv 1

                                                                                                                                        2

                                                                                                                                        1

                                                                                                                                        22

                                                                                                                                        1

                                                                                                                                        2 aavxx

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Parseval identity for

                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                        bull Pythagorean theorem

                                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                                        bull ANOVA - sums of squares

                                                                                                                                        x

                                                                                                                                        nvv 1

                                                                                                                                        2

                                                                                                                                        1

                                                                                                                                        22

                                                                                                                                        1

                                                                                                                                        2 aavxx

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Parseval identity for

                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                        bull Pythagorean theorem

                                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                                        bull ANOVA - sums of squares

                                                                                                                                        bull Transform has same length as

                                                                                                                                        ie ldquorotation in rdquo

                                                                                                                                        x

                                                                                                                                        nvv 1

                                                                                                                                        2

                                                                                                                                        1

                                                                                                                                        22

                                                                                                                                        1

                                                                                                                                        2 aavxx

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        a xd

                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                        xV

                                                                                                                                        V

                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                        bull Find that Solves

                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                        xV

                                                                                                                                        V

                                                                                                                                        VxPV vxVv

                                                                                                                                        min

                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                        bull Find that Solves

                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                        bull For Inner Product (Hilbert) Space

                                                                                                                                        Exists and is Unique

                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                        xV

                                                                                                                                        V

                                                                                                                                        VxPV vxVv

                                                                                                                                        min

                                                                                                                                        xPV

                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        d VB

                                                                                                                                        xBBBBxP tVV

                                                                                                                                        tVVV

                                                                                                                                        1

                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        d VB

                                                                                                                                        xBBBBxP tVV

                                                                                                                                        tVVV

                                                                                                                                        1

                                                                                                                                        tVV

                                                                                                                                        tVVV BBBBP

                                                                                                                                        1

                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                        (note same operation underlies least squares)

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        d VB

                                                                                                                                        xBBBBxP tVV

                                                                                                                                        tVVV

                                                                                                                                        1

                                                                                                                                        tVV

                                                                                                                                        tVVV BBBBP

                                                                                                                                        1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                        nnVtV IBB

                                                                                                                                        10

                                                                                                                                        01

                                                                                                                                        1

                                                                                                                                        111

                                                                                                                                        1

                                                                                                                                        1

                                                                                                                                        nnn

                                                                                                                                        n

                                                                                                                                        ntn

                                                                                                                                        t

                                                                                                                                        vvvv

                                                                                                                                        vvvv

                                                                                                                                        vv

                                                                                                                                        v

                                                                                                                                        v

                                                                                                                                        nvv 1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                        bull So =

                                                                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                        (Recall Right Multrsquon)

                                                                                                                                        nnVtV IBB

                                                                                                                                        10

                                                                                                                                        01

                                                                                                                                        1

                                                                                                                                        111

                                                                                                                                        1

                                                                                                                                        1

                                                                                                                                        nnn

                                                                                                                                        n

                                                                                                                                        ntn

                                                                                                                                        t

                                                                                                                                        vvvv

                                                                                                                                        vvvv

                                                                                                                                        vv

                                                                                                                                        v

                                                                                                                                        v

                                                                                                                                        xBBxP tVVV

                                                                                                                                        x V

                                                                                                                                        nvv 1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                        and

                                                                                                                                        V

                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                        and

                                                                                                                                        bull Parseval Inequality

                                                                                                                                        V

                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                        2

                                                                                                                                        1

                                                                                                                                        22

                                                                                                                                        1

                                                                                                                                        22 aavxxxP

                                                                                                                                        n

                                                                                                                                        ii

                                                                                                                                        n

                                                                                                                                        iiV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                        (So All of Above Applies)

                                                                                                                                        ddU IUU t

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                        (So All of Above Applies)

                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                        ddU IUU t

                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                        i ii 2

                                                                                                                                        1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                        (So All of Above Applies)

                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                        ddU IUU t

                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                        i ii 2

                                                                                                                                        1

                                                                                                                                        U

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                        For a Matrix

                                                                                                                                        Find

                                                                                                                                        ndX

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                        For a Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        with Entries

                                                                                                                                        called Singular Values

                                                                                                                                        ndX

                                                                                                                                        ndS

                                                                                                                                        )min(1 ndss

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                        For a Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        with Entries

                                                                                                                                        called Singular Values

                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                        (recall )

                                                                                                                                        ndX

                                                                                                                                        ndS

                                                                                                                                        )min(1 ndss

                                                                                                                                        ddU nnV

                                                                                                                                        IVVUU tt

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                        For a Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        with Entries

                                                                                                                                        called Singular Values

                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                        (recall )

                                                                                                                                        So That

                                                                                                                                        ndX

                                                                                                                                        ndS

                                                                                                                                        )min(1 ndss

                                                                                                                                        ddU nnV

                                                                                                                                        IVVUU tt tUSVX

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                        bull First rotate

                                                                                                                                        X

                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                        bull First rotate

                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                        X

                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                        is

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                        bull First rotate

                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                        bull Third rotate again

                                                                                                                                        X

                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                        is

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                        bull First rotate

                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                        bull Third rotate again

                                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                                        X

                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                        is

                                                                                                                                        SVD Compact Representation

                                                                                                                                        Useful Labeling

                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        )min(1 dnss

                                                                                                                                        SVD Compact Representation

                                                                                                                                        Useful Labeling

                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        )min(1 dnss

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        r

                                                                                                                                        SVD Compact Representation

                                                                                                                                        Useful Labeling

                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                        Let = of positive singular values

                                                                                                                                        )min(1 dnss

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        r

                                                                                                                                        SVD Compact Representation

                                                                                                                                        Useful Labeling

                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                        Let = of positive singular values

                                                                                                                                        Then

                                                                                                                                        Where are truncations of

                                                                                                                                        trnrrrd VSUX

                                                                                                                                        VSU

                                                                                                                                        )min(1 dnss

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Full Representation

                                                                                                                                        =

                                                                                                                                        Graphics Display Assumes

                                                                                                                                        ndX ddU ndS nn

                                                                                                                                        tV

                                                                                                                                        nd

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Full Representation

                                                                                                                                        =

                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                        ndX ddU ndS nn

                                                                                                                                        tV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Full Representation

                                                                                                                                        =

                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                        All 0s in Bottom

                                                                                                                                        ndX ddU ndS nn

                                                                                                                                        tV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Reduced Representation

                                                                                                                                        =

                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                        ndX ddU nnS nn

                                                                                                                                        tV

                                                                                                                                        nnd 0

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Reduced Representation

                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                        tV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Reduced Representation

                                                                                                                                        =

                                                                                                                                        Also Some of These May be 0

                                                                                                                                        ndX ndU nnS nn

                                                                                                                                        tV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Compact Representation

                                                                                                                                        = ndX rdU

                                                                                                                                        rrS nrtV

                                                                                                                                        0

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Compact Representation

                                                                                                                                        =

                                                                                                                                        These Get 0ed Out

                                                                                                                                        ndX rdU

                                                                                                                                        rrS nrtV

                                                                                                                                        0

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        SVD Compact Representation

                                                                                                                                        = ndX rdU

                                                                                                                                        rrS nrtV

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                        Find

                                                                                                                                        ddX

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        Called Eigenvalues

                                                                                                                                        Convenient Ordering

                                                                                                                                        ddX

                                                                                                                                        d

                                                                                                                                        D

                                                                                                                                        0

                                                                                                                                        01

                                                                                                                                        n 1

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                        (ie )

                                                                                                                                        ddX

                                                                                                                                        d

                                                                                                                                        D

                                                                                                                                        0

                                                                                                                                        01

                                                                                                                                        ddB

                                                                                                                                        ddtt IBBBB

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                        (ie )

                                                                                                                                        So that ie

                                                                                                                                        ddX

                                                                                                                                        d

                                                                                                                                        D

                                                                                                                                        0

                                                                                                                                        01

                                                                                                                                        ddB

                                                                                                                                        ddtt IBBBB

                                                                                                                                        DBBX tBDBX

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                        (looks similar)

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                        Complex (uses )

                                                                                                                                        VU

                                                                                                                                        1i

                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                        VU

                                                                                                                                        X

                                                                                                                                        BVU

                                                                                                                                        1i

                                                                                                                                        • Participant Presentations
                                                                                                                                        • PCA to find clusters
                                                                                                                                        • Statistical Smoothing
                                                                                                                                        • Kernel Density Estimation
                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                        • SiZer Background
                                                                                                                                        • SiZer Background (2)
                                                                                                                                        • SiZer Background (3)
                                                                                                                                        • SiZer Background (4)
                                                                                                                                        • SiZer Background (5)
                                                                                                                                        • SiZer Overview
                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                        • New Subpopulation View
                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                        • Detailed Look at PCA
                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                        • An Interesting Historical Note
                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                        • Review of Linear Algebra
                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Norm of a Vector (cont)

                                                                                                                                          bull Length Normalized Vector

                                                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                          amp is a Direction Vector)

                                                                                                                                          x

                                                                                                                                          x

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Norm of a Vector (cont)

                                                                                                                                          bull Length Normalized Vector

                                                                                                                                          (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                          amp is a Direction Vector)

                                                                                                                                          bull Define Distance as

                                                                                                                                          x

                                                                                                                                          x

                                                                                                                                          yxyxyxyxd t

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Inner (Dot Scalar) Product

                                                                                                                                          bull for Vectors and

                                                                                                                                          yxyxyx td

                                                                                                                                          jjj

                                                                                                                                          1

                                                                                                                                          x y

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Inner (Dot Scalar) Product

                                                                                                                                          bull for Vectors and

                                                                                                                                          bull Related to Norm via

                                                                                                                                          yxyxyx td

                                                                                                                                          jjj

                                                                                                                                          1

                                                                                                                                          x y

                                                                                                                                          21 xxx

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                                                          yyxx

                                                                                                                                          yx

                                                                                                                                          yx

                                                                                                                                          yxyxangle

                                                                                                                                          tt

                                                                                                                                          t

                                                                                                                                          11 cos

                                                                                                                                          cos

                                                                                                                                          x y

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Inner (Dot Scalar) Product (cont)

                                                                                                                                          bull measures ldquoangle between and rdquo as

                                                                                                                                          bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                          if and only if

                                                                                                                                          yyxx

                                                                                                                                          yx

                                                                                                                                          yx

                                                                                                                                          yxyxangle

                                                                                                                                          tt

                                                                                                                                          t

                                                                                                                                          11 cos

                                                                                                                                          cos

                                                                                                                                          x y

                                                                                                                                          yx 0 yx

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis

                                                                                                                                          bull All Orthogonal to each other

                                                                                                                                          ie for

                                                                                                                                          bull All have Length 1

                                                                                                                                          ie for

                                                                                                                                          nvv 1

                                                                                                                                          1 ii vv

                                                                                                                                          0 ii vv ii

                                                                                                                                          ni 1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                          bull Spectral Representation

                                                                                                                                          where

                                                                                                                                          nvv 1

                                                                                                                                          n

                                                                                                                                          iii vax

                                                                                                                                          1

                                                                                                                                          ii vxa

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                          bull Spectral Representation

                                                                                                                                          where

                                                                                                                                          (Coefficient is Inner Product Cool Notation)

                                                                                                                                          nvv 1

                                                                                                                                          n

                                                                                                                                          iii vax

                                                                                                                                          1

                                                                                                                                          ii vxa

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                          bull Spectral Representation

                                                                                                                                          where

                                                                                                                                          Check

                                                                                                                                          nvv 1

                                                                                                                                          n

                                                                                                                                          iii vax

                                                                                                                                          1

                                                                                                                                          ii vxa

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iiii avvavvavx

                                                                                                                                          1

                                                                                                                                          1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                          bull Spectral Representation

                                                                                                                                          where

                                                                                                                                          Check

                                                                                                                                          bull Matrix Notation where ie

                                                                                                                                          For the Basis Matrix

                                                                                                                                          nvv 1

                                                                                                                                          n

                                                                                                                                          iii vax

                                                                                                                                          1

                                                                                                                                          ii vxa

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iiii avvavvavx

                                                                                                                                          1

                                                                                                                                          1

                                                                                                                                          aBx Bxa tt xBa t

                                                                                                                                          nvvB 1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                          bull Spectral Representation

                                                                                                                                          where

                                                                                                                                          Check

                                                                                                                                          bull Matrix Notation where ie

                                                                                                                                          is called transform of

                                                                                                                                          (eg Fourier or Wavelet)

                                                                                                                                          nvv 1

                                                                                                                                          n

                                                                                                                                          iii vax

                                                                                                                                          1

                                                                                                                                          ii vxa

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iii

                                                                                                                                          n

                                                                                                                                          iiii avvavvavx

                                                                                                                                          1

                                                                                                                                          1

                                                                                                                                          aBx Bxa tt xBa t

                                                                                                                                          xa

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Parseval identity for

                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                          x

                                                                                                                                          nvv 1

                                                                                                                                          2

                                                                                                                                          1

                                                                                                                                          22

                                                                                                                                          1

                                                                                                                                          2 aavxx

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Parseval identity for

                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                          bull Pythagorean theorem

                                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                                          bull ANOVA - sums of squares

                                                                                                                                          x

                                                                                                                                          nvv 1

                                                                                                                                          2

                                                                                                                                          1

                                                                                                                                          22

                                                                                                                                          1

                                                                                                                                          2 aavxx

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Parseval identity for

                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                          bull Pythagorean theorem

                                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                                          bull ANOVA - sums of squares

                                                                                                                                          bull Transform has same length as

                                                                                                                                          ie ldquorotation in rdquo

                                                                                                                                          x

                                                                                                                                          nvv 1

                                                                                                                                          2

                                                                                                                                          1

                                                                                                                                          22

                                                                                                                                          1

                                                                                                                                          2 aavxx

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          a xd

                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                          xV

                                                                                                                                          V

                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                          bull Find that Solves

                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                          xV

                                                                                                                                          V

                                                                                                                                          VxPV vxVv

                                                                                                                                          min

                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                          bull Find that Solves

                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                          bull For Inner Product (Hilbert) Space

                                                                                                                                          Exists and is Unique

                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                          xV

                                                                                                                                          V

                                                                                                                                          VxPV vxVv

                                                                                                                                          min

                                                                                                                                          xPV

                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          d VB

                                                                                                                                          xBBBBxP tVV

                                                                                                                                          tVVV

                                                                                                                                          1

                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          d VB

                                                                                                                                          xBBBBxP tVV

                                                                                                                                          tVVV

                                                                                                                                          1

                                                                                                                                          tVV

                                                                                                                                          tVVV BBBBP

                                                                                                                                          1

                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                          (note same operation underlies least squares)

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          d VB

                                                                                                                                          xBBBBxP tVV

                                                                                                                                          tVVV

                                                                                                                                          1

                                                                                                                                          tVV

                                                                                                                                          tVVV BBBBP

                                                                                                                                          1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                          nnVtV IBB

                                                                                                                                          10

                                                                                                                                          01

                                                                                                                                          1

                                                                                                                                          111

                                                                                                                                          1

                                                                                                                                          1

                                                                                                                                          nnn

                                                                                                                                          n

                                                                                                                                          ntn

                                                                                                                                          t

                                                                                                                                          vvvv

                                                                                                                                          vvvv

                                                                                                                                          vv

                                                                                                                                          v

                                                                                                                                          v

                                                                                                                                          nvv 1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                          bull So =

                                                                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                          (Recall Right Multrsquon)

                                                                                                                                          nnVtV IBB

                                                                                                                                          10

                                                                                                                                          01

                                                                                                                                          1

                                                                                                                                          111

                                                                                                                                          1

                                                                                                                                          1

                                                                                                                                          nnn

                                                                                                                                          n

                                                                                                                                          ntn

                                                                                                                                          t

                                                                                                                                          vvvv

                                                                                                                                          vvvv

                                                                                                                                          vv

                                                                                                                                          v

                                                                                                                                          v

                                                                                                                                          xBBxP tVVV

                                                                                                                                          x V

                                                                                                                                          nvv 1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                          and

                                                                                                                                          V

                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                          and

                                                                                                                                          bull Parseval Inequality

                                                                                                                                          V

                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                          2

                                                                                                                                          1

                                                                                                                                          22

                                                                                                                                          1

                                                                                                                                          22 aavxxxP

                                                                                                                                          n

                                                                                                                                          ii

                                                                                                                                          n

                                                                                                                                          iiV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                          (So All of Above Applies)

                                                                                                                                          ddU IUU t

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                          (So All of Above Applies)

                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                          ddU IUU t

                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                          i ii 2

                                                                                                                                          1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                          (So All of Above Applies)

                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                          ddU IUU t

                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                          i ii 2

                                                                                                                                          1

                                                                                                                                          U

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                          For a Matrix

                                                                                                                                          Find

                                                                                                                                          ndX

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                          For a Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          with Entries

                                                                                                                                          called Singular Values

                                                                                                                                          ndX

                                                                                                                                          ndS

                                                                                                                                          )min(1 ndss

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                          For a Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          with Entries

                                                                                                                                          called Singular Values

                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                          (recall )

                                                                                                                                          ndX

                                                                                                                                          ndS

                                                                                                                                          )min(1 ndss

                                                                                                                                          ddU nnV

                                                                                                                                          IVVUU tt

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                          For a Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          with Entries

                                                                                                                                          called Singular Values

                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                          (recall )

                                                                                                                                          So That

                                                                                                                                          ndX

                                                                                                                                          ndS

                                                                                                                                          )min(1 ndss

                                                                                                                                          ddU nnV

                                                                                                                                          IVVUU tt tUSVX

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                          bull First rotate

                                                                                                                                          X

                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                          bull First rotate

                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                          X

                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                          is

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                          bull First rotate

                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                          bull Third rotate again

                                                                                                                                          X

                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                          is

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                          bull First rotate

                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                          bull Third rotate again

                                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                                          X

                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                          is

                                                                                                                                          SVD Compact Representation

                                                                                                                                          Useful Labeling

                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          )min(1 dnss

                                                                                                                                          SVD Compact Representation

                                                                                                                                          Useful Labeling

                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          )min(1 dnss

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          r

                                                                                                                                          SVD Compact Representation

                                                                                                                                          Useful Labeling

                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                          Let = of positive singular values

                                                                                                                                          )min(1 dnss

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          r

                                                                                                                                          SVD Compact Representation

                                                                                                                                          Useful Labeling

                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                          Let = of positive singular values

                                                                                                                                          Then

                                                                                                                                          Where are truncations of

                                                                                                                                          trnrrrd VSUX

                                                                                                                                          VSU

                                                                                                                                          )min(1 dnss

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Full Representation

                                                                                                                                          =

                                                                                                                                          Graphics Display Assumes

                                                                                                                                          ndX ddU ndS nn

                                                                                                                                          tV

                                                                                                                                          nd

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Full Representation

                                                                                                                                          =

                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                          ndX ddU ndS nn

                                                                                                                                          tV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Full Representation

                                                                                                                                          =

                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                          All 0s in Bottom

                                                                                                                                          ndX ddU ndS nn

                                                                                                                                          tV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Reduced Representation

                                                                                                                                          =

                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                          ndX ddU nnS nn

                                                                                                                                          tV

                                                                                                                                          nnd 0

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Reduced Representation

                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                          tV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Reduced Representation

                                                                                                                                          =

                                                                                                                                          Also Some of These May be 0

                                                                                                                                          ndX ndU nnS nn

                                                                                                                                          tV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Compact Representation

                                                                                                                                          = ndX rdU

                                                                                                                                          rrS nrtV

                                                                                                                                          0

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Compact Representation

                                                                                                                                          =

                                                                                                                                          These Get 0ed Out

                                                                                                                                          ndX rdU

                                                                                                                                          rrS nrtV

                                                                                                                                          0

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          SVD Compact Representation

                                                                                                                                          = ndX rdU

                                                                                                                                          rrS nrtV

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                          Find

                                                                                                                                          ddX

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          Called Eigenvalues

                                                                                                                                          Convenient Ordering

                                                                                                                                          ddX

                                                                                                                                          d

                                                                                                                                          D

                                                                                                                                          0

                                                                                                                                          01

                                                                                                                                          n 1

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                          (ie )

                                                                                                                                          ddX

                                                                                                                                          d

                                                                                                                                          D

                                                                                                                                          0

                                                                                                                                          01

                                                                                                                                          ddB

                                                                                                                                          ddtt IBBBB

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                          (ie )

                                                                                                                                          So that ie

                                                                                                                                          ddX

                                                                                                                                          d

                                                                                                                                          D

                                                                                                                                          0

                                                                                                                                          01

                                                                                                                                          ddB

                                                                                                                                          ddtt IBBBB

                                                                                                                                          DBBX tBDBX

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                          (looks similar)

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                          Complex (uses )

                                                                                                                                          VU

                                                                                                                                          1i

                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                          VU

                                                                                                                                          X

                                                                                                                                          BVU

                                                                                                                                          1i

                                                                                                                                          • Participant Presentations
                                                                                                                                          • PCA to find clusters
                                                                                                                                          • Statistical Smoothing
                                                                                                                                          • Kernel Density Estimation
                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                          • SiZer Background
                                                                                                                                          • SiZer Background (2)
                                                                                                                                          • SiZer Background (3)
                                                                                                                                          • SiZer Background (4)
                                                                                                                                          • SiZer Background (5)
                                                                                                                                          • SiZer Overview
                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                          • New Subpopulation View
                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                          • Detailed Look at PCA
                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                          • An Interesting Historical Note
                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                          • Review of Linear Algebra
                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Norm of a Vector (cont)

                                                                                                                                            bull Length Normalized Vector

                                                                                                                                            (has Length 1 thus on Surf of Unit Sphere

                                                                                                                                            amp is a Direction Vector)

                                                                                                                                            bull Define Distance as

                                                                                                                                            x

                                                                                                                                            x

                                                                                                                                            yxyxyxyxd t

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Inner (Dot Scalar) Product

                                                                                                                                            bull for Vectors and

                                                                                                                                            yxyxyx td

                                                                                                                                            jjj

                                                                                                                                            1

                                                                                                                                            x y

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Inner (Dot Scalar) Product

                                                                                                                                            bull for Vectors and

                                                                                                                                            bull Related to Norm via

                                                                                                                                            yxyxyx td

                                                                                                                                            jjj

                                                                                                                                            1

                                                                                                                                            x y

                                                                                                                                            21 xxx

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                                                            yyxx

                                                                                                                                            yx

                                                                                                                                            yx

                                                                                                                                            yxyxangle

                                                                                                                                            tt

                                                                                                                                            t

                                                                                                                                            11 cos

                                                                                                                                            cos

                                                                                                                                            x y

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Inner (Dot Scalar) Product (cont)

                                                                                                                                            bull measures ldquoangle between and rdquo as

                                                                                                                                            bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                            if and only if

                                                                                                                                            yyxx

                                                                                                                                            yx

                                                                                                                                            yx

                                                                                                                                            yxyxangle

                                                                                                                                            tt

                                                                                                                                            t

                                                                                                                                            11 cos

                                                                                                                                            cos

                                                                                                                                            x y

                                                                                                                                            yx 0 yx

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis

                                                                                                                                            bull All Orthogonal to each other

                                                                                                                                            ie for

                                                                                                                                            bull All have Length 1

                                                                                                                                            ie for

                                                                                                                                            nvv 1

                                                                                                                                            1 ii vv

                                                                                                                                            0 ii vv ii

                                                                                                                                            ni 1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                            bull Spectral Representation

                                                                                                                                            where

                                                                                                                                            nvv 1

                                                                                                                                            n

                                                                                                                                            iii vax

                                                                                                                                            1

                                                                                                                                            ii vxa

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                            bull Spectral Representation

                                                                                                                                            where

                                                                                                                                            (Coefficient is Inner Product Cool Notation)

                                                                                                                                            nvv 1

                                                                                                                                            n

                                                                                                                                            iii vax

                                                                                                                                            1

                                                                                                                                            ii vxa

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                            bull Spectral Representation

                                                                                                                                            where

                                                                                                                                            Check

                                                                                                                                            nvv 1

                                                                                                                                            n

                                                                                                                                            iii vax

                                                                                                                                            1

                                                                                                                                            ii vxa

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iiii avvavvavx

                                                                                                                                            1

                                                                                                                                            1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                            bull Spectral Representation

                                                                                                                                            where

                                                                                                                                            Check

                                                                                                                                            bull Matrix Notation where ie

                                                                                                                                            For the Basis Matrix

                                                                                                                                            nvv 1

                                                                                                                                            n

                                                                                                                                            iii vax

                                                                                                                                            1

                                                                                                                                            ii vxa

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iiii avvavvavx

                                                                                                                                            1

                                                                                                                                            1

                                                                                                                                            aBx Bxa tt xBa t

                                                                                                                                            nvvB 1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                            bull Spectral Representation

                                                                                                                                            where

                                                                                                                                            Check

                                                                                                                                            bull Matrix Notation where ie

                                                                                                                                            is called transform of

                                                                                                                                            (eg Fourier or Wavelet)

                                                                                                                                            nvv 1

                                                                                                                                            n

                                                                                                                                            iii vax

                                                                                                                                            1

                                                                                                                                            ii vxa

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iii

                                                                                                                                            n

                                                                                                                                            iiii avvavvavx

                                                                                                                                            1

                                                                                                                                            1

                                                                                                                                            aBx Bxa tt xBa t

                                                                                                                                            xa

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Parseval identity for

                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                            x

                                                                                                                                            nvv 1

                                                                                                                                            2

                                                                                                                                            1

                                                                                                                                            22

                                                                                                                                            1

                                                                                                                                            2 aavxx

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Parseval identity for

                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                            bull Pythagorean theorem

                                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                                            bull ANOVA - sums of squares

                                                                                                                                            x

                                                                                                                                            nvv 1

                                                                                                                                            2

                                                                                                                                            1

                                                                                                                                            22

                                                                                                                                            1

                                                                                                                                            2 aavxx

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Parseval identity for

                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                            bull Pythagorean theorem

                                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                                            bull ANOVA - sums of squares

                                                                                                                                            bull Transform has same length as

                                                                                                                                            ie ldquorotation in rdquo

                                                                                                                                            x

                                                                                                                                            nvv 1

                                                                                                                                            2

                                                                                                                                            1

                                                                                                                                            22

                                                                                                                                            1

                                                                                                                                            2 aavxx

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            a xd

                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                            xV

                                                                                                                                            V

                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                            bull Find that Solves

                                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                            xV

                                                                                                                                            V

                                                                                                                                            VxPV vxVv

                                                                                                                                            min

                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                            bull Find that Solves

                                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                                            bull For Inner Product (Hilbert) Space

                                                                                                                                            Exists and is Unique

                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                            xV

                                                                                                                                            V

                                                                                                                                            VxPV vxVv

                                                                                                                                            min

                                                                                                                                            xPV

                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            d VB

                                                                                                                                            xBBBBxP tVV

                                                                                                                                            tVVV

                                                                                                                                            1

                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            d VB

                                                                                                                                            xBBBBxP tVV

                                                                                                                                            tVVV

                                                                                                                                            1

                                                                                                                                            tVV

                                                                                                                                            tVVV BBBBP

                                                                                                                                            1

                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                            (note same operation underlies least squares)

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            d VB

                                                                                                                                            xBBBBxP tVV

                                                                                                                                            tVVV

                                                                                                                                            1

                                                                                                                                            tVV

                                                                                                                                            tVVV BBBBP

                                                                                                                                            1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                            nnVtV IBB

                                                                                                                                            10

                                                                                                                                            01

                                                                                                                                            1

                                                                                                                                            111

                                                                                                                                            1

                                                                                                                                            1

                                                                                                                                            nnn

                                                                                                                                            n

                                                                                                                                            ntn

                                                                                                                                            t

                                                                                                                                            vvvv

                                                                                                                                            vvvv

                                                                                                                                            vv

                                                                                                                                            v

                                                                                                                                            v

                                                                                                                                            nvv 1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                            bull So =

                                                                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                            (Recall Right Multrsquon)

                                                                                                                                            nnVtV IBB

                                                                                                                                            10

                                                                                                                                            01

                                                                                                                                            1

                                                                                                                                            111

                                                                                                                                            1

                                                                                                                                            1

                                                                                                                                            nnn

                                                                                                                                            n

                                                                                                                                            ntn

                                                                                                                                            t

                                                                                                                                            vvvv

                                                                                                                                            vvvv

                                                                                                                                            vv

                                                                                                                                            v

                                                                                                                                            v

                                                                                                                                            xBBxP tVVV

                                                                                                                                            x V

                                                                                                                                            nvv 1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                            and

                                                                                                                                            V

                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                            and

                                                                                                                                            bull Parseval Inequality

                                                                                                                                            V

                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                            2

                                                                                                                                            1

                                                                                                                                            22

                                                                                                                                            1

                                                                                                                                            22 aavxxxP

                                                                                                                                            n

                                                                                                                                            ii

                                                                                                                                            n

                                                                                                                                            iiV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                            (So All of Above Applies)

                                                                                                                                            ddU IUU t

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                            (So All of Above Applies)

                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                            ddU IUU t

                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                            i ii 2

                                                                                                                                            1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                            (So All of Above Applies)

                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                            ddU IUU t

                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                            i ii 2

                                                                                                                                            1

                                                                                                                                            U

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                            For a Matrix

                                                                                                                                            Find

                                                                                                                                            ndX

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                            For a Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            with Entries

                                                                                                                                            called Singular Values

                                                                                                                                            ndX

                                                                                                                                            ndS

                                                                                                                                            )min(1 ndss

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                            For a Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            with Entries

                                                                                                                                            called Singular Values

                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                            (recall )

                                                                                                                                            ndX

                                                                                                                                            ndS

                                                                                                                                            )min(1 ndss

                                                                                                                                            ddU nnV

                                                                                                                                            IVVUU tt

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                            For a Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            with Entries

                                                                                                                                            called Singular Values

                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                            (recall )

                                                                                                                                            So That

                                                                                                                                            ndX

                                                                                                                                            ndS

                                                                                                                                            )min(1 ndss

                                                                                                                                            ddU nnV

                                                                                                                                            IVVUU tt tUSVX

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                            bull First rotate

                                                                                                                                            X

                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                            bull First rotate

                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                            X

                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                            is

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                            bull First rotate

                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                            bull Third rotate again

                                                                                                                                            X

                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                            is

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                            bull First rotate

                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                            bull Third rotate again

                                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                                            X

                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                            is

                                                                                                                                            SVD Compact Representation

                                                                                                                                            Useful Labeling

                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            )min(1 dnss

                                                                                                                                            SVD Compact Representation

                                                                                                                                            Useful Labeling

                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            )min(1 dnss

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            r

                                                                                                                                            SVD Compact Representation

                                                                                                                                            Useful Labeling

                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                            Let = of positive singular values

                                                                                                                                            )min(1 dnss

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            r

                                                                                                                                            SVD Compact Representation

                                                                                                                                            Useful Labeling

                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                            Let = of positive singular values

                                                                                                                                            Then

                                                                                                                                            Where are truncations of

                                                                                                                                            trnrrrd VSUX

                                                                                                                                            VSU

                                                                                                                                            )min(1 dnss

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Full Representation

                                                                                                                                            =

                                                                                                                                            Graphics Display Assumes

                                                                                                                                            ndX ddU ndS nn

                                                                                                                                            tV

                                                                                                                                            nd

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Full Representation

                                                                                                                                            =

                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                            ndX ddU ndS nn

                                                                                                                                            tV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Full Representation

                                                                                                                                            =

                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                            All 0s in Bottom

                                                                                                                                            ndX ddU ndS nn

                                                                                                                                            tV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Reduced Representation

                                                                                                                                            =

                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                            ndX ddU nnS nn

                                                                                                                                            tV

                                                                                                                                            nnd 0

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Reduced Representation

                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                            tV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Reduced Representation

                                                                                                                                            =

                                                                                                                                            Also Some of These May be 0

                                                                                                                                            ndX ndU nnS nn

                                                                                                                                            tV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Compact Representation

                                                                                                                                            = ndX rdU

                                                                                                                                            rrS nrtV

                                                                                                                                            0

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Compact Representation

                                                                                                                                            =

                                                                                                                                            These Get 0ed Out

                                                                                                                                            ndX rdU

                                                                                                                                            rrS nrtV

                                                                                                                                            0

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            SVD Compact Representation

                                                                                                                                            = ndX rdU

                                                                                                                                            rrS nrtV

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                            Find

                                                                                                                                            ddX

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            Called Eigenvalues

                                                                                                                                            Convenient Ordering

                                                                                                                                            ddX

                                                                                                                                            d

                                                                                                                                            D

                                                                                                                                            0

                                                                                                                                            01

                                                                                                                                            n 1

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                            (ie )

                                                                                                                                            ddX

                                                                                                                                            d

                                                                                                                                            D

                                                                                                                                            0

                                                                                                                                            01

                                                                                                                                            ddB

                                                                                                                                            ddtt IBBBB

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                            (ie )

                                                                                                                                            So that ie

                                                                                                                                            ddX

                                                                                                                                            d

                                                                                                                                            D

                                                                                                                                            0

                                                                                                                                            01

                                                                                                                                            ddB

                                                                                                                                            ddtt IBBBB

                                                                                                                                            DBBX tBDBX

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                            (looks similar)

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                            Complex (uses )

                                                                                                                                            VU

                                                                                                                                            1i

                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                            VU

                                                                                                                                            X

                                                                                                                                            BVU

                                                                                                                                            1i

                                                                                                                                            • Participant Presentations
                                                                                                                                            • PCA to find clusters
                                                                                                                                            • Statistical Smoothing
                                                                                                                                            • Kernel Density Estimation
                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                            • SiZer Background
                                                                                                                                            • SiZer Background (2)
                                                                                                                                            • SiZer Background (3)
                                                                                                                                            • SiZer Background (4)
                                                                                                                                            • SiZer Background (5)
                                                                                                                                            • SiZer Overview
                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                            • New Subpopulation View
                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                            • Detailed Look at PCA
                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                            • An Interesting Historical Note
                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                            • Review of Linear Algebra
                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Inner (Dot Scalar) Product

                                                                                                                                              bull for Vectors and

                                                                                                                                              yxyxyx td

                                                                                                                                              jjj

                                                                                                                                              1

                                                                                                                                              x y

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Inner (Dot Scalar) Product

                                                                                                                                              bull for Vectors and

                                                                                                                                              bull Related to Norm via

                                                                                                                                              yxyxyx td

                                                                                                                                              jjj

                                                                                                                                              1

                                                                                                                                              x y

                                                                                                                                              21 xxx

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                                                              yyxx

                                                                                                                                              yx

                                                                                                                                              yx

                                                                                                                                              yxyxangle

                                                                                                                                              tt

                                                                                                                                              t

                                                                                                                                              11 cos

                                                                                                                                              cos

                                                                                                                                              x y

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Inner (Dot Scalar) Product (cont)

                                                                                                                                              bull measures ldquoangle between and rdquo as

                                                                                                                                              bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                              if and only if

                                                                                                                                              yyxx

                                                                                                                                              yx

                                                                                                                                              yx

                                                                                                                                              yxyxangle

                                                                                                                                              tt

                                                                                                                                              t

                                                                                                                                              11 cos

                                                                                                                                              cos

                                                                                                                                              x y

                                                                                                                                              yx 0 yx

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis

                                                                                                                                              bull All Orthogonal to each other

                                                                                                                                              ie for

                                                                                                                                              bull All have Length 1

                                                                                                                                              ie for

                                                                                                                                              nvv 1

                                                                                                                                              1 ii vv

                                                                                                                                              0 ii vv ii

                                                                                                                                              ni 1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                              bull Spectral Representation

                                                                                                                                              where

                                                                                                                                              nvv 1

                                                                                                                                              n

                                                                                                                                              iii vax

                                                                                                                                              1

                                                                                                                                              ii vxa

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                              bull Spectral Representation

                                                                                                                                              where

                                                                                                                                              (Coefficient is Inner Product Cool Notation)

                                                                                                                                              nvv 1

                                                                                                                                              n

                                                                                                                                              iii vax

                                                                                                                                              1

                                                                                                                                              ii vxa

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                              bull Spectral Representation

                                                                                                                                              where

                                                                                                                                              Check

                                                                                                                                              nvv 1

                                                                                                                                              n

                                                                                                                                              iii vax

                                                                                                                                              1

                                                                                                                                              ii vxa

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iiii avvavvavx

                                                                                                                                              1

                                                                                                                                              1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                              bull Spectral Representation

                                                                                                                                              where

                                                                                                                                              Check

                                                                                                                                              bull Matrix Notation where ie

                                                                                                                                              For the Basis Matrix

                                                                                                                                              nvv 1

                                                                                                                                              n

                                                                                                                                              iii vax

                                                                                                                                              1

                                                                                                                                              ii vxa

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iiii avvavvavx

                                                                                                                                              1

                                                                                                                                              1

                                                                                                                                              aBx Bxa tt xBa t

                                                                                                                                              nvvB 1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                              bull Spectral Representation

                                                                                                                                              where

                                                                                                                                              Check

                                                                                                                                              bull Matrix Notation where ie

                                                                                                                                              is called transform of

                                                                                                                                              (eg Fourier or Wavelet)

                                                                                                                                              nvv 1

                                                                                                                                              n

                                                                                                                                              iii vax

                                                                                                                                              1

                                                                                                                                              ii vxa

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iii

                                                                                                                                              n

                                                                                                                                              iiii avvavvavx

                                                                                                                                              1

                                                                                                                                              1

                                                                                                                                              aBx Bxa tt xBa t

                                                                                                                                              xa

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Parseval identity for

                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                              x

                                                                                                                                              nvv 1

                                                                                                                                              2

                                                                                                                                              1

                                                                                                                                              22

                                                                                                                                              1

                                                                                                                                              2 aavxx

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Parseval identity for

                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                              bull Pythagorean theorem

                                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                                              bull ANOVA - sums of squares

                                                                                                                                              x

                                                                                                                                              nvv 1

                                                                                                                                              2

                                                                                                                                              1

                                                                                                                                              22

                                                                                                                                              1

                                                                                                                                              2 aavxx

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Parseval identity for

                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                              bull Pythagorean theorem

                                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                                              bull ANOVA - sums of squares

                                                                                                                                              bull Transform has same length as

                                                                                                                                              ie ldquorotation in rdquo

                                                                                                                                              x

                                                                                                                                              nvv 1

                                                                                                                                              2

                                                                                                                                              1

                                                                                                                                              22

                                                                                                                                              1

                                                                                                                                              2 aavxx

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              a xd

                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                              xV

                                                                                                                                              V

                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                              bull Find that Solves

                                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                              xV

                                                                                                                                              V

                                                                                                                                              VxPV vxVv

                                                                                                                                              min

                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                              bull Find that Solves

                                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                                              bull For Inner Product (Hilbert) Space

                                                                                                                                              Exists and is Unique

                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                              xV

                                                                                                                                              V

                                                                                                                                              VxPV vxVv

                                                                                                                                              min

                                                                                                                                              xPV

                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              d VB

                                                                                                                                              xBBBBxP tVV

                                                                                                                                              tVVV

                                                                                                                                              1

                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              d VB

                                                                                                                                              xBBBBxP tVV

                                                                                                                                              tVVV

                                                                                                                                              1

                                                                                                                                              tVV

                                                                                                                                              tVVV BBBBP

                                                                                                                                              1

                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                              (note same operation underlies least squares)

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              d VB

                                                                                                                                              xBBBBxP tVV

                                                                                                                                              tVVV

                                                                                                                                              1

                                                                                                                                              tVV

                                                                                                                                              tVVV BBBBP

                                                                                                                                              1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                              nnVtV IBB

                                                                                                                                              10

                                                                                                                                              01

                                                                                                                                              1

                                                                                                                                              111

                                                                                                                                              1

                                                                                                                                              1

                                                                                                                                              nnn

                                                                                                                                              n

                                                                                                                                              ntn

                                                                                                                                              t

                                                                                                                                              vvvv

                                                                                                                                              vvvv

                                                                                                                                              vv

                                                                                                                                              v

                                                                                                                                              v

                                                                                                                                              nvv 1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                              bull So =

                                                                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                              (Recall Right Multrsquon)

                                                                                                                                              nnVtV IBB

                                                                                                                                              10

                                                                                                                                              01

                                                                                                                                              1

                                                                                                                                              111

                                                                                                                                              1

                                                                                                                                              1

                                                                                                                                              nnn

                                                                                                                                              n

                                                                                                                                              ntn

                                                                                                                                              t

                                                                                                                                              vvvv

                                                                                                                                              vvvv

                                                                                                                                              vv

                                                                                                                                              v

                                                                                                                                              v

                                                                                                                                              xBBxP tVVV

                                                                                                                                              x V

                                                                                                                                              nvv 1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                              and

                                                                                                                                              V

                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                              and

                                                                                                                                              bull Parseval Inequality

                                                                                                                                              V

                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                              2

                                                                                                                                              1

                                                                                                                                              22

                                                                                                                                              1

                                                                                                                                              22 aavxxxP

                                                                                                                                              n

                                                                                                                                              ii

                                                                                                                                              n

                                                                                                                                              iiV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                              (So All of Above Applies)

                                                                                                                                              ddU IUU t

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                              (So All of Above Applies)

                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                              ddU IUU t

                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                              i ii 2

                                                                                                                                              1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                              (So All of Above Applies)

                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                              ddU IUU t

                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                              i ii 2

                                                                                                                                              1

                                                                                                                                              U

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                              For a Matrix

                                                                                                                                              Find

                                                                                                                                              ndX

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                              For a Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              with Entries

                                                                                                                                              called Singular Values

                                                                                                                                              ndX

                                                                                                                                              ndS

                                                                                                                                              )min(1 ndss

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                              For a Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              with Entries

                                                                                                                                              called Singular Values

                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                              (recall )

                                                                                                                                              ndX

                                                                                                                                              ndS

                                                                                                                                              )min(1 ndss

                                                                                                                                              ddU nnV

                                                                                                                                              IVVUU tt

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                              For a Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              with Entries

                                                                                                                                              called Singular Values

                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                              (recall )

                                                                                                                                              So That

                                                                                                                                              ndX

                                                                                                                                              ndS

                                                                                                                                              )min(1 ndss

                                                                                                                                              ddU nnV

                                                                                                                                              IVVUU tt tUSVX

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                              bull First rotate

                                                                                                                                              X

                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                              bull First rotate

                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                              X

                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                              is

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                              bull First rotate

                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                              bull Third rotate again

                                                                                                                                              X

                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                              is

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                              bull First rotate

                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                              bull Third rotate again

                                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                                              X

                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                              is

                                                                                                                                              SVD Compact Representation

                                                                                                                                              Useful Labeling

                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              )min(1 dnss

                                                                                                                                              SVD Compact Representation

                                                                                                                                              Useful Labeling

                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              )min(1 dnss

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              r

                                                                                                                                              SVD Compact Representation

                                                                                                                                              Useful Labeling

                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                              Let = of positive singular values

                                                                                                                                              )min(1 dnss

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              r

                                                                                                                                              SVD Compact Representation

                                                                                                                                              Useful Labeling

                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                              Let = of positive singular values

                                                                                                                                              Then

                                                                                                                                              Where are truncations of

                                                                                                                                              trnrrrd VSUX

                                                                                                                                              VSU

                                                                                                                                              )min(1 dnss

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Full Representation

                                                                                                                                              =

                                                                                                                                              Graphics Display Assumes

                                                                                                                                              ndX ddU ndS nn

                                                                                                                                              tV

                                                                                                                                              nd

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Full Representation

                                                                                                                                              =

                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                              ndX ddU ndS nn

                                                                                                                                              tV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Full Representation

                                                                                                                                              =

                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                              All 0s in Bottom

                                                                                                                                              ndX ddU ndS nn

                                                                                                                                              tV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Reduced Representation

                                                                                                                                              =

                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                              ndX ddU nnS nn

                                                                                                                                              tV

                                                                                                                                              nnd 0

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Reduced Representation

                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                              tV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Reduced Representation

                                                                                                                                              =

                                                                                                                                              Also Some of These May be 0

                                                                                                                                              ndX ndU nnS nn

                                                                                                                                              tV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Compact Representation

                                                                                                                                              = ndX rdU

                                                                                                                                              rrS nrtV

                                                                                                                                              0

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Compact Representation

                                                                                                                                              =

                                                                                                                                              These Get 0ed Out

                                                                                                                                              ndX rdU

                                                                                                                                              rrS nrtV

                                                                                                                                              0

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              SVD Compact Representation

                                                                                                                                              = ndX rdU

                                                                                                                                              rrS nrtV

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                              Find

                                                                                                                                              ddX

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              Called Eigenvalues

                                                                                                                                              Convenient Ordering

                                                                                                                                              ddX

                                                                                                                                              d

                                                                                                                                              D

                                                                                                                                              0

                                                                                                                                              01

                                                                                                                                              n 1

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                              (ie )

                                                                                                                                              ddX

                                                                                                                                              d

                                                                                                                                              D

                                                                                                                                              0

                                                                                                                                              01

                                                                                                                                              ddB

                                                                                                                                              ddtt IBBBB

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                              (ie )

                                                                                                                                              So that ie

                                                                                                                                              ddX

                                                                                                                                              d

                                                                                                                                              D

                                                                                                                                              0

                                                                                                                                              01

                                                                                                                                              ddB

                                                                                                                                              ddtt IBBBB

                                                                                                                                              DBBX tBDBX

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                              (looks similar)

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                              Complex (uses )

                                                                                                                                              VU

                                                                                                                                              1i

                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                              VU

                                                                                                                                              X

                                                                                                                                              BVU

                                                                                                                                              1i

                                                                                                                                              • Participant Presentations
                                                                                                                                              • PCA to find clusters
                                                                                                                                              • Statistical Smoothing
                                                                                                                                              • Kernel Density Estimation
                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                              • SiZer Background
                                                                                                                                              • SiZer Background (2)
                                                                                                                                              • SiZer Background (3)
                                                                                                                                              • SiZer Background (4)
                                                                                                                                              • SiZer Background (5)
                                                                                                                                              • SiZer Overview
                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                              • New Subpopulation View
                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                              • Detailed Look at PCA
                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                              • An Interesting Historical Note
                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                              • Review of Linear Algebra
                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Inner (Dot Scalar) Product

                                                                                                                                                bull for Vectors and

                                                                                                                                                bull Related to Norm via

                                                                                                                                                yxyxyx td

                                                                                                                                                jjj

                                                                                                                                                1

                                                                                                                                                x y

                                                                                                                                                21 xxx

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                                                yyxx

                                                                                                                                                yx

                                                                                                                                                yx

                                                                                                                                                yxyxangle

                                                                                                                                                tt

                                                                                                                                                t

                                                                                                                                                11 cos

                                                                                                                                                cos

                                                                                                                                                x y

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Inner (Dot Scalar) Product (cont)

                                                                                                                                                bull measures ldquoangle between and rdquo as

                                                                                                                                                bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                                if and only if

                                                                                                                                                yyxx

                                                                                                                                                yx

                                                                                                                                                yx

                                                                                                                                                yxyxangle

                                                                                                                                                tt

                                                                                                                                                t

                                                                                                                                                11 cos

                                                                                                                                                cos

                                                                                                                                                x y

                                                                                                                                                yx 0 yx

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis

                                                                                                                                                bull All Orthogonal to each other

                                                                                                                                                ie for

                                                                                                                                                bull All have Length 1

                                                                                                                                                ie for

                                                                                                                                                nvv 1

                                                                                                                                                1 ii vv

                                                                                                                                                0 ii vv ii

                                                                                                                                                ni 1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                bull Spectral Representation

                                                                                                                                                where

                                                                                                                                                nvv 1

                                                                                                                                                n

                                                                                                                                                iii vax

                                                                                                                                                1

                                                                                                                                                ii vxa

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                bull Spectral Representation

                                                                                                                                                where

                                                                                                                                                (Coefficient is Inner Product Cool Notation)

                                                                                                                                                nvv 1

                                                                                                                                                n

                                                                                                                                                iii vax

                                                                                                                                                1

                                                                                                                                                ii vxa

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                bull Spectral Representation

                                                                                                                                                where

                                                                                                                                                Check

                                                                                                                                                nvv 1

                                                                                                                                                n

                                                                                                                                                iii vax

                                                                                                                                                1

                                                                                                                                                ii vxa

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iiii avvavvavx

                                                                                                                                                1

                                                                                                                                                1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                bull Spectral Representation

                                                                                                                                                where

                                                                                                                                                Check

                                                                                                                                                bull Matrix Notation where ie

                                                                                                                                                For the Basis Matrix

                                                                                                                                                nvv 1

                                                                                                                                                n

                                                                                                                                                iii vax

                                                                                                                                                1

                                                                                                                                                ii vxa

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iiii avvavvavx

                                                                                                                                                1

                                                                                                                                                1

                                                                                                                                                aBx Bxa tt xBa t

                                                                                                                                                nvvB 1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                bull Spectral Representation

                                                                                                                                                where

                                                                                                                                                Check

                                                                                                                                                bull Matrix Notation where ie

                                                                                                                                                is called transform of

                                                                                                                                                (eg Fourier or Wavelet)

                                                                                                                                                nvv 1

                                                                                                                                                n

                                                                                                                                                iii vax

                                                                                                                                                1

                                                                                                                                                ii vxa

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iii

                                                                                                                                                n

                                                                                                                                                iiii avvavvavx

                                                                                                                                                1

                                                                                                                                                1

                                                                                                                                                aBx Bxa tt xBa t

                                                                                                                                                xa

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Parseval identity for

                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                x

                                                                                                                                                nvv 1

                                                                                                                                                2

                                                                                                                                                1

                                                                                                                                                22

                                                                                                                                                1

                                                                                                                                                2 aavxx

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Parseval identity for

                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                bull Pythagorean theorem

                                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                                x

                                                                                                                                                nvv 1

                                                                                                                                                2

                                                                                                                                                1

                                                                                                                                                22

                                                                                                                                                1

                                                                                                                                                2 aavxx

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Parseval identity for

                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                bull Pythagorean theorem

                                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                                bull Transform has same length as

                                                                                                                                                ie ldquorotation in rdquo

                                                                                                                                                x

                                                                                                                                                nvv 1

                                                                                                                                                2

                                                                                                                                                1

                                                                                                                                                22

                                                                                                                                                1

                                                                                                                                                2 aavxx

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                a xd

                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                xV

                                                                                                                                                V

                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                bull Find that Solves

                                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                xV

                                                                                                                                                V

                                                                                                                                                VxPV vxVv

                                                                                                                                                min

                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                bull Find that Solves

                                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                                bull For Inner Product (Hilbert) Space

                                                                                                                                                Exists and is Unique

                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                xV

                                                                                                                                                V

                                                                                                                                                VxPV vxVv

                                                                                                                                                min

                                                                                                                                                xPV

                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                d VB

                                                                                                                                                xBBBBxP tVV

                                                                                                                                                tVVV

                                                                                                                                                1

                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                d VB

                                                                                                                                                xBBBBxP tVV

                                                                                                                                                tVVV

                                                                                                                                                1

                                                                                                                                                tVV

                                                                                                                                                tVVV BBBBP

                                                                                                                                                1

                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                (note same operation underlies least squares)

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                d VB

                                                                                                                                                xBBBBxP tVV

                                                                                                                                                tVVV

                                                                                                                                                1

                                                                                                                                                tVV

                                                                                                                                                tVVV BBBBP

                                                                                                                                                1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                nnVtV IBB

                                                                                                                                                10

                                                                                                                                                01

                                                                                                                                                1

                                                                                                                                                111

                                                                                                                                                1

                                                                                                                                                1

                                                                                                                                                nnn

                                                                                                                                                n

                                                                                                                                                ntn

                                                                                                                                                t

                                                                                                                                                vvvv

                                                                                                                                                vvvv

                                                                                                                                                vv

                                                                                                                                                v

                                                                                                                                                v

                                                                                                                                                nvv 1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                bull So =

                                                                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                (Recall Right Multrsquon)

                                                                                                                                                nnVtV IBB

                                                                                                                                                10

                                                                                                                                                01

                                                                                                                                                1

                                                                                                                                                111

                                                                                                                                                1

                                                                                                                                                1

                                                                                                                                                nnn

                                                                                                                                                n

                                                                                                                                                ntn

                                                                                                                                                t

                                                                                                                                                vvvv

                                                                                                                                                vvvv

                                                                                                                                                vv

                                                                                                                                                v

                                                                                                                                                v

                                                                                                                                                xBBxP tVVV

                                                                                                                                                x V

                                                                                                                                                nvv 1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                and

                                                                                                                                                V

                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                and

                                                                                                                                                bull Parseval Inequality

                                                                                                                                                V

                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                2

                                                                                                                                                1

                                                                                                                                                22

                                                                                                                                                1

                                                                                                                                                22 aavxxxP

                                                                                                                                                n

                                                                                                                                                ii

                                                                                                                                                n

                                                                                                                                                iiV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                (So All of Above Applies)

                                                                                                                                                ddU IUU t

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                (So All of Above Applies)

                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                ddU IUU t

                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                i ii 2

                                                                                                                                                1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                (So All of Above Applies)

                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                ddU IUU t

                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                i ii 2

                                                                                                                                                1

                                                                                                                                                U

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                For a Matrix

                                                                                                                                                Find

                                                                                                                                                ndX

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                For a Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                with Entries

                                                                                                                                                called Singular Values

                                                                                                                                                ndX

                                                                                                                                                ndS

                                                                                                                                                )min(1 ndss

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                For a Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                with Entries

                                                                                                                                                called Singular Values

                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                (recall )

                                                                                                                                                ndX

                                                                                                                                                ndS

                                                                                                                                                )min(1 ndss

                                                                                                                                                ddU nnV

                                                                                                                                                IVVUU tt

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                For a Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                with Entries

                                                                                                                                                called Singular Values

                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                (recall )

                                                                                                                                                So That

                                                                                                                                                ndX

                                                                                                                                                ndS

                                                                                                                                                )min(1 ndss

                                                                                                                                                ddU nnV

                                                                                                                                                IVVUU tt tUSVX

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                bull First rotate

                                                                                                                                                X

                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                bull First rotate

                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                X

                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                is

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                bull First rotate

                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                bull Third rotate again

                                                                                                                                                X

                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                is

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                bull First rotate

                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                bull Third rotate again

                                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                                X

                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                is

                                                                                                                                                SVD Compact Representation

                                                                                                                                                Useful Labeling

                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                )min(1 dnss

                                                                                                                                                SVD Compact Representation

                                                                                                                                                Useful Labeling

                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                )min(1 dnss

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                r

                                                                                                                                                SVD Compact Representation

                                                                                                                                                Useful Labeling

                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                Let = of positive singular values

                                                                                                                                                )min(1 dnss

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                r

                                                                                                                                                SVD Compact Representation

                                                                                                                                                Useful Labeling

                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                Let = of positive singular values

                                                                                                                                                Then

                                                                                                                                                Where are truncations of

                                                                                                                                                trnrrrd VSUX

                                                                                                                                                VSU

                                                                                                                                                )min(1 dnss

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Full Representation

                                                                                                                                                =

                                                                                                                                                Graphics Display Assumes

                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                tV

                                                                                                                                                nd

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Full Representation

                                                                                                                                                =

                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                tV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Full Representation

                                                                                                                                                =

                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                All 0s in Bottom

                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                tV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Reduced Representation

                                                                                                                                                =

                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                tV

                                                                                                                                                nnd 0

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Reduced Representation

                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                tV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Reduced Representation

                                                                                                                                                =

                                                                                                                                                Also Some of These May be 0

                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                tV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Compact Representation

                                                                                                                                                = ndX rdU

                                                                                                                                                rrS nrtV

                                                                                                                                                0

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Compact Representation

                                                                                                                                                =

                                                                                                                                                These Get 0ed Out

                                                                                                                                                ndX rdU

                                                                                                                                                rrS nrtV

                                                                                                                                                0

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                SVD Compact Representation

                                                                                                                                                = ndX rdU

                                                                                                                                                rrS nrtV

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                Find

                                                                                                                                                ddX

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                Called Eigenvalues

                                                                                                                                                Convenient Ordering

                                                                                                                                                ddX

                                                                                                                                                d

                                                                                                                                                D

                                                                                                                                                0

                                                                                                                                                01

                                                                                                                                                n 1

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                (ie )

                                                                                                                                                ddX

                                                                                                                                                d

                                                                                                                                                D

                                                                                                                                                0

                                                                                                                                                01

                                                                                                                                                ddB

                                                                                                                                                ddtt IBBBB

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                (ie )

                                                                                                                                                So that ie

                                                                                                                                                ddX

                                                                                                                                                d

                                                                                                                                                D

                                                                                                                                                0

                                                                                                                                                01

                                                                                                                                                ddB

                                                                                                                                                ddtt IBBBB

                                                                                                                                                DBBX tBDBX

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                (looks similar)

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                Complex (uses )

                                                                                                                                                VU

                                                                                                                                                1i

                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                VU

                                                                                                                                                X

                                                                                                                                                BVU

                                                                                                                                                1i

                                                                                                                                                • Participant Presentations
                                                                                                                                                • PCA to find clusters
                                                                                                                                                • Statistical Smoothing
                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                • SiZer Background
                                                                                                                                                • SiZer Background (2)
                                                                                                                                                • SiZer Background (3)
                                                                                                                                                • SiZer Background (4)
                                                                                                                                                • SiZer Background (5)
                                                                                                                                                • SiZer Overview
                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                • New Subpopulation View
                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                                                  yyxx

                                                                                                                                                  yx

                                                                                                                                                  yx

                                                                                                                                                  yxyxangle

                                                                                                                                                  tt

                                                                                                                                                  t

                                                                                                                                                  11 cos

                                                                                                                                                  cos

                                                                                                                                                  x y

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Inner (Dot Scalar) Product (cont)

                                                                                                                                                  bull measures ldquoangle between and rdquo as

                                                                                                                                                  bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                                  if and only if

                                                                                                                                                  yyxx

                                                                                                                                                  yx

                                                                                                                                                  yx

                                                                                                                                                  yxyxangle

                                                                                                                                                  tt

                                                                                                                                                  t

                                                                                                                                                  11 cos

                                                                                                                                                  cos

                                                                                                                                                  x y

                                                                                                                                                  yx 0 yx

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis

                                                                                                                                                  bull All Orthogonal to each other

                                                                                                                                                  ie for

                                                                                                                                                  bull All have Length 1

                                                                                                                                                  ie for

                                                                                                                                                  nvv 1

                                                                                                                                                  1 ii vv

                                                                                                                                                  0 ii vv ii

                                                                                                                                                  ni 1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                                  bull Spectral Representation

                                                                                                                                                  where

                                                                                                                                                  nvv 1

                                                                                                                                                  n

                                                                                                                                                  iii vax

                                                                                                                                                  1

                                                                                                                                                  ii vxa

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                                  bull Spectral Representation

                                                                                                                                                  where

                                                                                                                                                  (Coefficient is Inner Product Cool Notation)

                                                                                                                                                  nvv 1

                                                                                                                                                  n

                                                                                                                                                  iii vax

                                                                                                                                                  1

                                                                                                                                                  ii vxa

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                                  bull Spectral Representation

                                                                                                                                                  where

                                                                                                                                                  Check

                                                                                                                                                  nvv 1

                                                                                                                                                  n

                                                                                                                                                  iii vax

                                                                                                                                                  1

                                                                                                                                                  ii vxa

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iiii avvavvavx

                                                                                                                                                  1

                                                                                                                                                  1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                                  bull Spectral Representation

                                                                                                                                                  where

                                                                                                                                                  Check

                                                                                                                                                  bull Matrix Notation where ie

                                                                                                                                                  For the Basis Matrix

                                                                                                                                                  nvv 1

                                                                                                                                                  n

                                                                                                                                                  iii vax

                                                                                                                                                  1

                                                                                                                                                  ii vxa

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iiii avvavvavx

                                                                                                                                                  1

                                                                                                                                                  1

                                                                                                                                                  aBx Bxa tt xBa t

                                                                                                                                                  nvvB 1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Orthonormal Basis (cont)

                                                                                                                                                  bull Spectral Representation

                                                                                                                                                  where

                                                                                                                                                  Check

                                                                                                                                                  bull Matrix Notation where ie

                                                                                                                                                  is called transform of

                                                                                                                                                  (eg Fourier or Wavelet)

                                                                                                                                                  nvv 1

                                                                                                                                                  n

                                                                                                                                                  iii vax

                                                                                                                                                  1

                                                                                                                                                  ii vxa

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iii

                                                                                                                                                  n

                                                                                                                                                  iiii avvavvavx

                                                                                                                                                  1

                                                                                                                                                  1

                                                                                                                                                  aBx Bxa tt xBa t

                                                                                                                                                  xa

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Parseval identity for

                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                  x

                                                                                                                                                  nvv 1

                                                                                                                                                  2

                                                                                                                                                  1

                                                                                                                                                  22

                                                                                                                                                  1

                                                                                                                                                  2 aavxx

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Parseval identity for

                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                  bull Pythagorean theorem

                                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                                  x

                                                                                                                                                  nvv 1

                                                                                                                                                  2

                                                                                                                                                  1

                                                                                                                                                  22

                                                                                                                                                  1

                                                                                                                                                  2 aavxx

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Parseval identity for

                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                  bull Pythagorean theorem

                                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                                  bull Transform has same length as

                                                                                                                                                  ie ldquorotation in rdquo

                                                                                                                                                  x

                                                                                                                                                  nvv 1

                                                                                                                                                  2

                                                                                                                                                  1

                                                                                                                                                  22

                                                                                                                                                  1

                                                                                                                                                  2 aavxx

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  a xd

                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                  xV

                                                                                                                                                  V

                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                  bull Find that Solves

                                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                  xV

                                                                                                                                                  V

                                                                                                                                                  VxPV vxVv

                                                                                                                                                  min

                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                  bull Find that Solves

                                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                                  bull For Inner Product (Hilbert) Space

                                                                                                                                                  Exists and is Unique

                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                  xV

                                                                                                                                                  V

                                                                                                                                                  VxPV vxVv

                                                                                                                                                  min

                                                                                                                                                  xPV

                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  d VB

                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                  tVVV

                                                                                                                                                  1

                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  d VB

                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                  tVVV

                                                                                                                                                  1

                                                                                                                                                  tVV

                                                                                                                                                  tVVV BBBBP

                                                                                                                                                  1

                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                                  (note same operation underlies least squares)

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  d VB

                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                  tVVV

                                                                                                                                                  1

                                                                                                                                                  tVV

                                                                                                                                                  tVVV BBBBP

                                                                                                                                                  1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                  nnVtV IBB

                                                                                                                                                  10

                                                                                                                                                  01

                                                                                                                                                  1

                                                                                                                                                  111

                                                                                                                                                  1

                                                                                                                                                  1

                                                                                                                                                  nnn

                                                                                                                                                  n

                                                                                                                                                  ntn

                                                                                                                                                  t

                                                                                                                                                  vvvv

                                                                                                                                                  vvvv

                                                                                                                                                  vv

                                                                                                                                                  v

                                                                                                                                                  v

                                                                                                                                                  nvv 1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                  bull So =

                                                                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                  (Recall Right Multrsquon)

                                                                                                                                                  nnVtV IBB

                                                                                                                                                  10

                                                                                                                                                  01

                                                                                                                                                  1

                                                                                                                                                  111

                                                                                                                                                  1

                                                                                                                                                  1

                                                                                                                                                  nnn

                                                                                                                                                  n

                                                                                                                                                  ntn

                                                                                                                                                  t

                                                                                                                                                  vvvv

                                                                                                                                                  vvvv

                                                                                                                                                  vv

                                                                                                                                                  v

                                                                                                                                                  v

                                                                                                                                                  xBBxP tVVV

                                                                                                                                                  x V

                                                                                                                                                  nvv 1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                  and

                                                                                                                                                  V

                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                  and

                                                                                                                                                  bull Parseval Inequality

                                                                                                                                                  V

                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                  2

                                                                                                                                                  1

                                                                                                                                                  22

                                                                                                                                                  1

                                                                                                                                                  22 aavxxxP

                                                                                                                                                  n

                                                                                                                                                  ii

                                                                                                                                                  n

                                                                                                                                                  iiV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                  ddU IUU t

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                  ddU IUU t

                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                  i ii 2

                                                                                                                                                  1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                  ddU IUU t

                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                  i ii 2

                                                                                                                                                  1

                                                                                                                                                  U

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                  For a Matrix

                                                                                                                                                  Find

                                                                                                                                                  ndX

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                  For a Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  with Entries

                                                                                                                                                  called Singular Values

                                                                                                                                                  ndX

                                                                                                                                                  ndS

                                                                                                                                                  )min(1 ndss

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                  For a Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  with Entries

                                                                                                                                                  called Singular Values

                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                  (recall )

                                                                                                                                                  ndX

                                                                                                                                                  ndS

                                                                                                                                                  )min(1 ndss

                                                                                                                                                  ddU nnV

                                                                                                                                                  IVVUU tt

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                  For a Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  with Entries

                                                                                                                                                  called Singular Values

                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                  (recall )

                                                                                                                                                  So That

                                                                                                                                                  ndX

                                                                                                                                                  ndS

                                                                                                                                                  )min(1 ndss

                                                                                                                                                  ddU nnV

                                                                                                                                                  IVVUU tt tUSVX

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                  bull First rotate

                                                                                                                                                  X

                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                  bull First rotate

                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                  X

                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                  is

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                  bull First rotate

                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                  bull Third rotate again

                                                                                                                                                  X

                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                  is

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                  bull First rotate

                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                  bull Third rotate again

                                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                                  X

                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                  is

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  Useful Labeling

                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  )min(1 dnss

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  Useful Labeling

                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  )min(1 dnss

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  r

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  Useful Labeling

                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                  Let = of positive singular values

                                                                                                                                                  )min(1 dnss

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  r

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  Useful Labeling

                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                  Let = of positive singular values

                                                                                                                                                  Then

                                                                                                                                                  Where are truncations of

                                                                                                                                                  trnrrrd VSUX

                                                                                                                                                  VSU

                                                                                                                                                  )min(1 dnss

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Full Representation

                                                                                                                                                  =

                                                                                                                                                  Graphics Display Assumes

                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                  tV

                                                                                                                                                  nd

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Full Representation

                                                                                                                                                  =

                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                  tV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Full Representation

                                                                                                                                                  =

                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                  All 0s in Bottom

                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                  tV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                  =

                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                  tV

                                                                                                                                                  nnd 0

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                  tV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                  =

                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                  tV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  = ndX rdU

                                                                                                                                                  rrS nrtV

                                                                                                                                                  0

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  =

                                                                                                                                                  These Get 0ed Out

                                                                                                                                                  ndX rdU

                                                                                                                                                  rrS nrtV

                                                                                                                                                  0

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  SVD Compact Representation

                                                                                                                                                  = ndX rdU

                                                                                                                                                  rrS nrtV

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                  Find

                                                                                                                                                  ddX

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  Called Eigenvalues

                                                                                                                                                  Convenient Ordering

                                                                                                                                                  ddX

                                                                                                                                                  d

                                                                                                                                                  D

                                                                                                                                                  0

                                                                                                                                                  01

                                                                                                                                                  n 1

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                  (ie )

                                                                                                                                                  ddX

                                                                                                                                                  d

                                                                                                                                                  D

                                                                                                                                                  0

                                                                                                                                                  01

                                                                                                                                                  ddB

                                                                                                                                                  ddtt IBBBB

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                  (ie )

                                                                                                                                                  So that ie

                                                                                                                                                  ddX

                                                                                                                                                  d

                                                                                                                                                  D

                                                                                                                                                  0

                                                                                                                                                  01

                                                                                                                                                  ddB

                                                                                                                                                  ddtt IBBBB

                                                                                                                                                  DBBX tBDBX

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                  (looks similar)

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                  Complex (uses )

                                                                                                                                                  VU

                                                                                                                                                  1i

                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                  VU

                                                                                                                                                  X

                                                                                                                                                  BVU

                                                                                                                                                  1i

                                                                                                                                                  • Participant Presentations
                                                                                                                                                  • PCA to find clusters
                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                  • SiZer Background
                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                  • SiZer Overview
                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                  • New Subpopulation View
                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Inner (Dot Scalar) Product (cont)

                                                                                                                                                    bull measures ldquoangle between and rdquo as

                                                                                                                                                    bull key to Orthogonality ie Perpendiculrsquoty

                                                                                                                                                    if and only if

                                                                                                                                                    yyxx

                                                                                                                                                    yx

                                                                                                                                                    yx

                                                                                                                                                    yxyxangle

                                                                                                                                                    tt

                                                                                                                                                    t

                                                                                                                                                    11 cos

                                                                                                                                                    cos

                                                                                                                                                    x y

                                                                                                                                                    yx 0 yx

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis

                                                                                                                                                    bull All Orthogonal to each other

                                                                                                                                                    ie for

                                                                                                                                                    bull All have Length 1

                                                                                                                                                    ie for

                                                                                                                                                    nvv 1

                                                                                                                                                    1 ii vv

                                                                                                                                                    0 ii vv ii

                                                                                                                                                    ni 1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                                    bull Spectral Representation

                                                                                                                                                    where

                                                                                                                                                    nvv 1

                                                                                                                                                    n

                                                                                                                                                    iii vax

                                                                                                                                                    1

                                                                                                                                                    ii vxa

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                                    bull Spectral Representation

                                                                                                                                                    where

                                                                                                                                                    (Coefficient is Inner Product Cool Notation)

                                                                                                                                                    nvv 1

                                                                                                                                                    n

                                                                                                                                                    iii vax

                                                                                                                                                    1

                                                                                                                                                    ii vxa

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                                    bull Spectral Representation

                                                                                                                                                    where

                                                                                                                                                    Check

                                                                                                                                                    nvv 1

                                                                                                                                                    n

                                                                                                                                                    iii vax

                                                                                                                                                    1

                                                                                                                                                    ii vxa

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iiii avvavvavx

                                                                                                                                                    1

                                                                                                                                                    1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                                    bull Spectral Representation

                                                                                                                                                    where

                                                                                                                                                    Check

                                                                                                                                                    bull Matrix Notation where ie

                                                                                                                                                    For the Basis Matrix

                                                                                                                                                    nvv 1

                                                                                                                                                    n

                                                                                                                                                    iii vax

                                                                                                                                                    1

                                                                                                                                                    ii vxa

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iiii avvavvavx

                                                                                                                                                    1

                                                                                                                                                    1

                                                                                                                                                    aBx Bxa tt xBa t

                                                                                                                                                    nvvB 1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Orthonormal Basis (cont)

                                                                                                                                                    bull Spectral Representation

                                                                                                                                                    where

                                                                                                                                                    Check

                                                                                                                                                    bull Matrix Notation where ie

                                                                                                                                                    is called transform of

                                                                                                                                                    (eg Fourier or Wavelet)

                                                                                                                                                    nvv 1

                                                                                                                                                    n

                                                                                                                                                    iii vax

                                                                                                                                                    1

                                                                                                                                                    ii vxa

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iii

                                                                                                                                                    n

                                                                                                                                                    iiii avvavvavx

                                                                                                                                                    1

                                                                                                                                                    1

                                                                                                                                                    aBx Bxa tt xBa t

                                                                                                                                                    xa

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Parseval identity for

                                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                                    x

                                                                                                                                                    nvv 1

                                                                                                                                                    2

                                                                                                                                                    1

                                                                                                                                                    22

                                                                                                                                                    1

                                                                                                                                                    2 aavxx

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Parseval identity for

                                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                                    bull Pythagorean theorem

                                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                                    x

                                                                                                                                                    nvv 1

                                                                                                                                                    2

                                                                                                                                                    1

                                                                                                                                                    22

                                                                                                                                                    1

                                                                                                                                                    2 aavxx

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Parseval identity for

                                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                                    bull Pythagorean theorem

                                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                                    bull Transform has same length as

                                                                                                                                                    ie ldquorotation in rdquo

                                                                                                                                                    x

                                                                                                                                                    nvv 1

                                                                                                                                                    2

                                                                                                                                                    1

                                                                                                                                                    22

                                                                                                                                                    1

                                                                                                                                                    2 aavxx

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    a xd

                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                    xV

                                                                                                                                                    V

                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                    bull Find that Solves

                                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                    xV

                                                                                                                                                    V

                                                                                                                                                    VxPV vxVv

                                                                                                                                                    min

                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                    bull Find that Solves

                                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                                    bull For Inner Product (Hilbert) Space

                                                                                                                                                    Exists and is Unique

                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                    xV

                                                                                                                                                    V

                                                                                                                                                    VxPV vxVv

                                                                                                                                                    min

                                                                                                                                                    xPV

                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    d VB

                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                    tVVV

                                                                                                                                                    1

                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    d VB

                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                    tVVV

                                                                                                                                                    1

                                                                                                                                                    tVV

                                                                                                                                                    tVVV BBBBP

                                                                                                                                                    1

                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                                    (note same operation underlies least squares)

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    d VB

                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                    tVVV

                                                                                                                                                    1

                                                                                                                                                    tVV

                                                                                                                                                    tVVV BBBBP

                                                                                                                                                    1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                    nnVtV IBB

                                                                                                                                                    10

                                                                                                                                                    01

                                                                                                                                                    1

                                                                                                                                                    111

                                                                                                                                                    1

                                                                                                                                                    1

                                                                                                                                                    nnn

                                                                                                                                                    n

                                                                                                                                                    ntn

                                                                                                                                                    t

                                                                                                                                                    vvvv

                                                                                                                                                    vvvv

                                                                                                                                                    vv

                                                                                                                                                    v

                                                                                                                                                    v

                                                                                                                                                    nvv 1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                    bull So =

                                                                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                    (Recall Right Multrsquon)

                                                                                                                                                    nnVtV IBB

                                                                                                                                                    10

                                                                                                                                                    01

                                                                                                                                                    1

                                                                                                                                                    111

                                                                                                                                                    1

                                                                                                                                                    1

                                                                                                                                                    nnn

                                                                                                                                                    n

                                                                                                                                                    ntn

                                                                                                                                                    t

                                                                                                                                                    vvvv

                                                                                                                                                    vvvv

                                                                                                                                                    vv

                                                                                                                                                    v

                                                                                                                                                    v

                                                                                                                                                    xBBxP tVVV

                                                                                                                                                    x V

                                                                                                                                                    nvv 1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                    and

                                                                                                                                                    V

                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                    and

                                                                                                                                                    bull Parseval Inequality

                                                                                                                                                    V

                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                    2

                                                                                                                                                    1

                                                                                                                                                    22

                                                                                                                                                    1

                                                                                                                                                    22 aavxxxP

                                                                                                                                                    n

                                                                                                                                                    ii

                                                                                                                                                    n

                                                                                                                                                    iiV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                    ddU IUU t

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                    ddU IUU t

                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                    i ii 2

                                                                                                                                                    1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                    ddU IUU t

                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                    i ii 2

                                                                                                                                                    1

                                                                                                                                                    U

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                    For a Matrix

                                                                                                                                                    Find

                                                                                                                                                    ndX

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                    For a Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    with Entries

                                                                                                                                                    called Singular Values

                                                                                                                                                    ndX

                                                                                                                                                    ndS

                                                                                                                                                    )min(1 ndss

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                    For a Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    with Entries

                                                                                                                                                    called Singular Values

                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                    (recall )

                                                                                                                                                    ndX

                                                                                                                                                    ndS

                                                                                                                                                    )min(1 ndss

                                                                                                                                                    ddU nnV

                                                                                                                                                    IVVUU tt

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                    For a Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    with Entries

                                                                                                                                                    called Singular Values

                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                    (recall )

                                                                                                                                                    So That

                                                                                                                                                    ndX

                                                                                                                                                    ndS

                                                                                                                                                    )min(1 ndss

                                                                                                                                                    ddU nnV

                                                                                                                                                    IVVUU tt tUSVX

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                    bull First rotate

                                                                                                                                                    X

                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                    bull First rotate

                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                    X

                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                    is

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                    bull First rotate

                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                    bull Third rotate again

                                                                                                                                                    X

                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                    is

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                    bull First rotate

                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                    bull Third rotate again

                                                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                                                    X

                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                    is

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    Useful Labeling

                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    )min(1 dnss

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    Useful Labeling

                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    )min(1 dnss

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    r

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    Useful Labeling

                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                    Let = of positive singular values

                                                                                                                                                    )min(1 dnss

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    r

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    Useful Labeling

                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                    Let = of positive singular values

                                                                                                                                                    Then

                                                                                                                                                    Where are truncations of

                                                                                                                                                    trnrrrd VSUX

                                                                                                                                                    VSU

                                                                                                                                                    )min(1 dnss

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Full Representation

                                                                                                                                                    =

                                                                                                                                                    Graphics Display Assumes

                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                    tV

                                                                                                                                                    nd

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Full Representation

                                                                                                                                                    =

                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                    tV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Full Representation

                                                                                                                                                    =

                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                    All 0s in Bottom

                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                    tV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                    =

                                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                                    ndX ddU nnS nn

                                                                                                                                                    tV

                                                                                                                                                    nnd 0

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                    tV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                    =

                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                    tV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    = ndX rdU

                                                                                                                                                    rrS nrtV

                                                                                                                                                    0

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    =

                                                                                                                                                    These Get 0ed Out

                                                                                                                                                    ndX rdU

                                                                                                                                                    rrS nrtV

                                                                                                                                                    0

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    SVD Compact Representation

                                                                                                                                                    = ndX rdU

                                                                                                                                                    rrS nrtV

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                    Find

                                                                                                                                                    ddX

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    Called Eigenvalues

                                                                                                                                                    Convenient Ordering

                                                                                                                                                    ddX

                                                                                                                                                    d

                                                                                                                                                    D

                                                                                                                                                    0

                                                                                                                                                    01

                                                                                                                                                    n 1

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                    (ie )

                                                                                                                                                    ddX

                                                                                                                                                    d

                                                                                                                                                    D

                                                                                                                                                    0

                                                                                                                                                    01

                                                                                                                                                    ddB

                                                                                                                                                    ddtt IBBBB

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                    (ie )

                                                                                                                                                    So that ie

                                                                                                                                                    ddX

                                                                                                                                                    d

                                                                                                                                                    D

                                                                                                                                                    0

                                                                                                                                                    01

                                                                                                                                                    ddB

                                                                                                                                                    ddtt IBBBB

                                                                                                                                                    DBBX tBDBX

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                    (looks similar)

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                    Complex (uses )

                                                                                                                                                    VU

                                                                                                                                                    1i

                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                    VU

                                                                                                                                                    X

                                                                                                                                                    BVU

                                                                                                                                                    1i

                                                                                                                                                    • Participant Presentations
                                                                                                                                                    • PCA to find clusters
                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                    • SiZer Background
                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                    • SiZer Overview
                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                    • New Subpopulation View
                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis

                                                                                                                                                      bull All Orthogonal to each other

                                                                                                                                                      ie for

                                                                                                                                                      bull All have Length 1

                                                                                                                                                      ie for

                                                                                                                                                      nvv 1

                                                                                                                                                      1 ii vv

                                                                                                                                                      0 ii vv ii

                                                                                                                                                      ni 1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                                      bull Spectral Representation

                                                                                                                                                      where

                                                                                                                                                      nvv 1

                                                                                                                                                      n

                                                                                                                                                      iii vax

                                                                                                                                                      1

                                                                                                                                                      ii vxa

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                                      bull Spectral Representation

                                                                                                                                                      where

                                                                                                                                                      (Coefficient is Inner Product Cool Notation)

                                                                                                                                                      nvv 1

                                                                                                                                                      n

                                                                                                                                                      iii vax

                                                                                                                                                      1

                                                                                                                                                      ii vxa

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                                      bull Spectral Representation

                                                                                                                                                      where

                                                                                                                                                      Check

                                                                                                                                                      nvv 1

                                                                                                                                                      n

                                                                                                                                                      iii vax

                                                                                                                                                      1

                                                                                                                                                      ii vxa

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iiii avvavvavx

                                                                                                                                                      1

                                                                                                                                                      1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                                      bull Spectral Representation

                                                                                                                                                      where

                                                                                                                                                      Check

                                                                                                                                                      bull Matrix Notation where ie

                                                                                                                                                      For the Basis Matrix

                                                                                                                                                      nvv 1

                                                                                                                                                      n

                                                                                                                                                      iii vax

                                                                                                                                                      1

                                                                                                                                                      ii vxa

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iiii avvavvavx

                                                                                                                                                      1

                                                                                                                                                      1

                                                                                                                                                      aBx Bxa tt xBa t

                                                                                                                                                      nvvB 1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Orthonormal Basis (cont)

                                                                                                                                                      bull Spectral Representation

                                                                                                                                                      where

                                                                                                                                                      Check

                                                                                                                                                      bull Matrix Notation where ie

                                                                                                                                                      is called transform of

                                                                                                                                                      (eg Fourier or Wavelet)

                                                                                                                                                      nvv 1

                                                                                                                                                      n

                                                                                                                                                      iii vax

                                                                                                                                                      1

                                                                                                                                                      ii vxa

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iii

                                                                                                                                                      n

                                                                                                                                                      iiii avvavvavx

                                                                                                                                                      1

                                                                                                                                                      1

                                                                                                                                                      aBx Bxa tt xBa t

                                                                                                                                                      xa

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Parseval identity for

                                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                                      x

                                                                                                                                                      nvv 1

                                                                                                                                                      2

                                                                                                                                                      1

                                                                                                                                                      22

                                                                                                                                                      1

                                                                                                                                                      2 aavxx

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Parseval identity for

                                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                                      bull Pythagorean theorem

                                                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                                                      bull ANOVA - sums of squares

                                                                                                                                                      x

                                                                                                                                                      nvv 1

                                                                                                                                                      2

                                                                                                                                                      1

                                                                                                                                                      22

                                                                                                                                                      1

                                                                                                                                                      2 aavxx

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Parseval identity for

                                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                                      bull Pythagorean theorem

                                                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                                                      bull ANOVA - sums of squares

                                                                                                                                                      bull Transform has same length as

                                                                                                                                                      ie ldquorotation in rdquo

                                                                                                                                                      x

                                                                                                                                                      nvv 1

                                                                                                                                                      2

                                                                                                                                                      1

                                                                                                                                                      22

                                                                                                                                                      1

                                                                                                                                                      2 aavxx

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      a xd

                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                      xV

                                                                                                                                                      V

                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                      bull Find that Solves

                                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                      xV

                                                                                                                                                      V

                                                                                                                                                      VxPV vxVv

                                                                                                                                                      min

                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                      bull Find that Solves

                                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                                      bull For Inner Product (Hilbert) Space

                                                                                                                                                      Exists and is Unique

                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                      xV

                                                                                                                                                      V

                                                                                                                                                      VxPV vxVv

                                                                                                                                                      min

                                                                                                                                                      xPV

                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      d VB

                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                      tVVV

                                                                                                                                                      1

                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      d VB

                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                      tVVV

                                                                                                                                                      1

                                                                                                                                                      tVV

                                                                                                                                                      tVVV BBBBP

                                                                                                                                                      1

                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                                      (note same operation underlies least squares)

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      d VB

                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                      tVVV

                                                                                                                                                      1

                                                                                                                                                      tVV

                                                                                                                                                      tVVV BBBBP

                                                                                                                                                      1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                                      nnVtV IBB

                                                                                                                                                      10

                                                                                                                                                      01

                                                                                                                                                      1

                                                                                                                                                      111

                                                                                                                                                      1

                                                                                                                                                      1

                                                                                                                                                      nnn

                                                                                                                                                      n

                                                                                                                                                      ntn

                                                                                                                                                      t

                                                                                                                                                      vvvv

                                                                                                                                                      vvvv

                                                                                                                                                      vv

                                                                                                                                                      v

                                                                                                                                                      v

                                                                                                                                                      nvv 1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                                      bull So =

                                                                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                      (Recall Right Multrsquon)

                                                                                                                                                      nnVtV IBB

                                                                                                                                                      10

                                                                                                                                                      01

                                                                                                                                                      1

                                                                                                                                                      111

                                                                                                                                                      1

                                                                                                                                                      1

                                                                                                                                                      nnn

                                                                                                                                                      n

                                                                                                                                                      ntn

                                                                                                                                                      t

                                                                                                                                                      vvvv

                                                                                                                                                      vvvv

                                                                                                                                                      vv

                                                                                                                                                      v

                                                                                                                                                      v

                                                                                                                                                      xBBxP tVVV

                                                                                                                                                      x V

                                                                                                                                                      nvv 1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                      and

                                                                                                                                                      V

                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                      and

                                                                                                                                                      bull Parseval Inequality

                                                                                                                                                      V

                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                      2

                                                                                                                                                      1

                                                                                                                                                      22

                                                                                                                                                      1

                                                                                                                                                      22 aavxxxP

                                                                                                                                                      n

                                                                                                                                                      ii

                                                                                                                                                      n

                                                                                                                                                      iiV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                      ddU IUU t

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                      ddU IUU t

                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                      i ii 2

                                                                                                                                                      1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                      ddU IUU t

                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                      i ii 2

                                                                                                                                                      1

                                                                                                                                                      U

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                      For a Matrix

                                                                                                                                                      Find

                                                                                                                                                      ndX

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                      For a Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      with Entries

                                                                                                                                                      called Singular Values

                                                                                                                                                      ndX

                                                                                                                                                      ndS

                                                                                                                                                      )min(1 ndss

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                      For a Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      with Entries

                                                                                                                                                      called Singular Values

                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                      (recall )

                                                                                                                                                      ndX

                                                                                                                                                      ndS

                                                                                                                                                      )min(1 ndss

                                                                                                                                                      ddU nnV

                                                                                                                                                      IVVUU tt

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                      For a Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      with Entries

                                                                                                                                                      called Singular Values

                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                      (recall )

                                                                                                                                                      So That

                                                                                                                                                      ndX

                                                                                                                                                      ndS

                                                                                                                                                      )min(1 ndss

                                                                                                                                                      ddU nnV

                                                                                                                                                      IVVUU tt tUSVX

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                      bull First rotate

                                                                                                                                                      X

                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                      bull First rotate

                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                      X

                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                      is

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                      bull First rotate

                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                      bull Third rotate again

                                                                                                                                                      X

                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                      is

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                      bull First rotate

                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                      bull Third rotate again

                                                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                                                      X

                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                      is

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      Useful Labeling

                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      )min(1 dnss

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      Useful Labeling

                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      )min(1 dnss

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      r

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      Useful Labeling

                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                      Let = of positive singular values

                                                                                                                                                      )min(1 dnss

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      r

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      Useful Labeling

                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                      Let = of positive singular values

                                                                                                                                                      Then

                                                                                                                                                      Where are truncations of

                                                                                                                                                      trnrrrd VSUX

                                                                                                                                                      VSU

                                                                                                                                                      )min(1 dnss

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Full Representation

                                                                                                                                                      =

                                                                                                                                                      Graphics Display Assumes

                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                      tV

                                                                                                                                                      nd

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Full Representation

                                                                                                                                                      =

                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                      tV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Full Representation

                                                                                                                                                      =

                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                      All 0s in Bottom

                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                      tV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                      =

                                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                                      ndX ddU nnS nn

                                                                                                                                                      tV

                                                                                                                                                      nnd 0

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                                      tV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                      =

                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                      tV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      = ndX rdU

                                                                                                                                                      rrS nrtV

                                                                                                                                                      0

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      =

                                                                                                                                                      These Get 0ed Out

                                                                                                                                                      ndX rdU

                                                                                                                                                      rrS nrtV

                                                                                                                                                      0

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      SVD Compact Representation

                                                                                                                                                      = ndX rdU

                                                                                                                                                      rrS nrtV

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                      Find

                                                                                                                                                      ddX

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      Called Eigenvalues

                                                                                                                                                      Convenient Ordering

                                                                                                                                                      ddX

                                                                                                                                                      d

                                                                                                                                                      D

                                                                                                                                                      0

                                                                                                                                                      01

                                                                                                                                                      n 1

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                      (ie )

                                                                                                                                                      ddX

                                                                                                                                                      d

                                                                                                                                                      D

                                                                                                                                                      0

                                                                                                                                                      01

                                                                                                                                                      ddB

                                                                                                                                                      ddtt IBBBB

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                      (ie )

                                                                                                                                                      So that ie

                                                                                                                                                      ddX

                                                                                                                                                      d

                                                                                                                                                      D

                                                                                                                                                      0

                                                                                                                                                      01

                                                                                                                                                      ddB

                                                                                                                                                      ddtt IBBBB

                                                                                                                                                      DBBX tBDBX

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                      (looks similar)

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                      Complex (uses )

                                                                                                                                                      VU

                                                                                                                                                      1i

                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                      VU

                                                                                                                                                      X

                                                                                                                                                      BVU

                                                                                                                                                      1i

                                                                                                                                                      • Participant Presentations
                                                                                                                                                      • PCA to find clusters
                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                      • SiZer Background
                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                      • SiZer Overview
                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                      • New Subpopulation View
                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                                        bull Spectral Representation

                                                                                                                                                        where

                                                                                                                                                        nvv 1

                                                                                                                                                        n

                                                                                                                                                        iii vax

                                                                                                                                                        1

                                                                                                                                                        ii vxa

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                                        bull Spectral Representation

                                                                                                                                                        where

                                                                                                                                                        (Coefficient is Inner Product Cool Notation)

                                                                                                                                                        nvv 1

                                                                                                                                                        n

                                                                                                                                                        iii vax

                                                                                                                                                        1

                                                                                                                                                        ii vxa

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                                        bull Spectral Representation

                                                                                                                                                        where

                                                                                                                                                        Check

                                                                                                                                                        nvv 1

                                                                                                                                                        n

                                                                                                                                                        iii vax

                                                                                                                                                        1

                                                                                                                                                        ii vxa

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iiii avvavvavx

                                                                                                                                                        1

                                                                                                                                                        1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                                        bull Spectral Representation

                                                                                                                                                        where

                                                                                                                                                        Check

                                                                                                                                                        bull Matrix Notation where ie

                                                                                                                                                        For the Basis Matrix

                                                                                                                                                        nvv 1

                                                                                                                                                        n

                                                                                                                                                        iii vax

                                                                                                                                                        1

                                                                                                                                                        ii vxa

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iiii avvavvavx

                                                                                                                                                        1

                                                                                                                                                        1

                                                                                                                                                        aBx Bxa tt xBa t

                                                                                                                                                        nvvB 1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Orthonormal Basis (cont)

                                                                                                                                                        bull Spectral Representation

                                                                                                                                                        where

                                                                                                                                                        Check

                                                                                                                                                        bull Matrix Notation where ie

                                                                                                                                                        is called transform of

                                                                                                                                                        (eg Fourier or Wavelet)

                                                                                                                                                        nvv 1

                                                                                                                                                        n

                                                                                                                                                        iii vax

                                                                                                                                                        1

                                                                                                                                                        ii vxa

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iii

                                                                                                                                                        n

                                                                                                                                                        iiii avvavvavx

                                                                                                                                                        1

                                                                                                                                                        1

                                                                                                                                                        aBx Bxa tt xBa t

                                                                                                                                                        xa

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Parseval identity for

                                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                                        x

                                                                                                                                                        nvv 1

                                                                                                                                                        2

                                                                                                                                                        1

                                                                                                                                                        22

                                                                                                                                                        1

                                                                                                                                                        2 aavxx

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Parseval identity for

                                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                                        bull Pythagorean theorem

                                                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                                                        bull ANOVA - sums of squares

                                                                                                                                                        x

                                                                                                                                                        nvv 1

                                                                                                                                                        2

                                                                                                                                                        1

                                                                                                                                                        22

                                                                                                                                                        1

                                                                                                                                                        2 aavxx

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Parseval identity for

                                                                                                                                                        in subsp genrsquod by o n basis

                                                                                                                                                        bull Pythagorean theorem

                                                                                                                                                        bull ldquoDecomposition of Energyrdquo

                                                                                                                                                        bull ANOVA - sums of squares

                                                                                                                                                        bull Transform has same length as

                                                                                                                                                        ie ldquorotation in rdquo

                                                                                                                                                        x

                                                                                                                                                        nvv 1

                                                                                                                                                        2

                                                                                                                                                        1

                                                                                                                                                        22

                                                                                                                                                        1

                                                                                                                                                        2 aavxx

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        a xd

                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                        xV

                                                                                                                                                        V

                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                        bull Find that Solves

                                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                        xV

                                                                                                                                                        V

                                                                                                                                                        VxPV vxVv

                                                                                                                                                        min

                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                        bull Find that Solves

                                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                                        bull For Inner Product (Hilbert) Space

                                                                                                                                                        Exists and is Unique

                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                        xV

                                                                                                                                                        V

                                                                                                                                                        VxPV vxVv

                                                                                                                                                        min

                                                                                                                                                        xPV

                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        d VB

                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                        tVVV

                                                                                                                                                        1

                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        d VB

                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                        tVVV

                                                                                                                                                        1

                                                                                                                                                        tVV

                                                                                                                                                        tVVV BBBBP

                                                                                                                                                        1

                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                                        (note same operation underlies least squares)

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        d VB

                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                        tVVV

                                                                                                                                                        1

                                                                                                                                                        tVV

                                                                                                                                                        tVVV BBBBP

                                                                                                                                                        1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                                        nnVtV IBB

                                                                                                                                                        10

                                                                                                                                                        01

                                                                                                                                                        1

                                                                                                                                                        111

                                                                                                                                                        1

                                                                                                                                                        1

                                                                                                                                                        nnn

                                                                                                                                                        n

                                                                                                                                                        ntn

                                                                                                                                                        t

                                                                                                                                                        vvvv

                                                                                                                                                        vvvv

                                                                                                                                                        vv

                                                                                                                                                        v

                                                                                                                                                        v

                                                                                                                                                        nvv 1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                                        bull So =

                                                                                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                        (Recall Right Multrsquon)

                                                                                                                                                        nnVtV IBB

                                                                                                                                                        10

                                                                                                                                                        01

                                                                                                                                                        1

                                                                                                                                                        111

                                                                                                                                                        1

                                                                                                                                                        1

                                                                                                                                                        nnn

                                                                                                                                                        n

                                                                                                                                                        ntn

                                                                                                                                                        t

                                                                                                                                                        vvvv

                                                                                                                                                        vvvv

                                                                                                                                                        vv

                                                                                                                                                        v

                                                                                                                                                        v

                                                                                                                                                        xBBxP tVVV

                                                                                                                                                        x V

                                                                                                                                                        nvv 1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                        and

                                                                                                                                                        V

                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                        and

                                                                                                                                                        bull Parseval Inequality

                                                                                                                                                        V

                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                        2

                                                                                                                                                        1

                                                                                                                                                        22

                                                                                                                                                        1

                                                                                                                                                        22 aavxxxP

                                                                                                                                                        n

                                                                                                                                                        ii

                                                                                                                                                        n

                                                                                                                                                        iiV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                        ddU IUU t

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                        ddU IUU t

                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                        i ii 2

                                                                                                                                                        1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                        ddU IUU t

                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                        i ii 2

                                                                                                                                                        1

                                                                                                                                                        U

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                        For a Matrix

                                                                                                                                                        Find

                                                                                                                                                        ndX

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                        For a Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        with Entries

                                                                                                                                                        called Singular Values

                                                                                                                                                        ndX

                                                                                                                                                        ndS

                                                                                                                                                        )min(1 ndss

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                        For a Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        with Entries

                                                                                                                                                        called Singular Values

                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                        (recall )

                                                                                                                                                        ndX

                                                                                                                                                        ndS

                                                                                                                                                        )min(1 ndss

                                                                                                                                                        ddU nnV

                                                                                                                                                        IVVUU tt

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                        For a Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        with Entries

                                                                                                                                                        called Singular Values

                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                        (recall )

                                                                                                                                                        So That

                                                                                                                                                        ndX

                                                                                                                                                        ndS

                                                                                                                                                        )min(1 ndss

                                                                                                                                                        ddU nnV

                                                                                                                                                        IVVUU tt tUSVX

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                        bull First rotate

                                                                                                                                                        X

                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                        bull First rotate

                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                        X

                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                        is

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                        bull First rotate

                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                        bull Third rotate again

                                                                                                                                                        X

                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                        is

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                        bull First rotate

                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                        bull Third rotate again

                                                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                                                        X

                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                        is

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        Useful Labeling

                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        )min(1 dnss

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        Useful Labeling

                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        )min(1 dnss

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        r

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        Useful Labeling

                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                        Let = of positive singular values

                                                                                                                                                        )min(1 dnss

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        r

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        Useful Labeling

                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                        Let = of positive singular values

                                                                                                                                                        Then

                                                                                                                                                        Where are truncations of

                                                                                                                                                        trnrrrd VSUX

                                                                                                                                                        VSU

                                                                                                                                                        )min(1 dnss

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Full Representation

                                                                                                                                                        =

                                                                                                                                                        Graphics Display Assumes

                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                        tV

                                                                                                                                                        nd

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Full Representation

                                                                                                                                                        =

                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                        tV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Full Representation

                                                                                                                                                        =

                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                        All 0s in Bottom

                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                        tV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                        =

                                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                                        ndX ddU nnS nn

                                                                                                                                                        tV

                                                                                                                                                        nnd 0

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                                        tV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                        =

                                                                                                                                                        Also Some of These May be 0

                                                                                                                                                        ndX ndU nnS nn

                                                                                                                                                        tV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        = ndX rdU

                                                                                                                                                        rrS nrtV

                                                                                                                                                        0

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        =

                                                                                                                                                        These Get 0ed Out

                                                                                                                                                        ndX rdU

                                                                                                                                                        rrS nrtV

                                                                                                                                                        0

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        SVD Compact Representation

                                                                                                                                                        = ndX rdU

                                                                                                                                                        rrS nrtV

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                        Find

                                                                                                                                                        ddX

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        Called Eigenvalues

                                                                                                                                                        Convenient Ordering

                                                                                                                                                        ddX

                                                                                                                                                        d

                                                                                                                                                        D

                                                                                                                                                        0

                                                                                                                                                        01

                                                                                                                                                        n 1

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                        (ie )

                                                                                                                                                        ddX

                                                                                                                                                        d

                                                                                                                                                        D

                                                                                                                                                        0

                                                                                                                                                        01

                                                                                                                                                        ddB

                                                                                                                                                        ddtt IBBBB

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                        (ie )

                                                                                                                                                        So that ie

                                                                                                                                                        ddX

                                                                                                                                                        d

                                                                                                                                                        D

                                                                                                                                                        0

                                                                                                                                                        01

                                                                                                                                                        ddB

                                                                                                                                                        ddtt IBBBB

                                                                                                                                                        DBBX tBDBX

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                        (looks similar)

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                        Complex (uses )

                                                                                                                                                        VU

                                                                                                                                                        1i

                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                        VU

                                                                                                                                                        X

                                                                                                                                                        BVU

                                                                                                                                                        1i

                                                                                                                                                        • Participant Presentations
                                                                                                                                                        • PCA to find clusters
                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                        • SiZer Background
                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                        • SiZer Overview
                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                        • New Subpopulation View
                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                                          bull Spectral Representation

                                                                                                                                                          where

                                                                                                                                                          (Coefficient is Inner Product Cool Notation)

                                                                                                                                                          nvv 1

                                                                                                                                                          n

                                                                                                                                                          iii vax

                                                                                                                                                          1

                                                                                                                                                          ii vxa

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                                          bull Spectral Representation

                                                                                                                                                          where

                                                                                                                                                          Check

                                                                                                                                                          nvv 1

                                                                                                                                                          n

                                                                                                                                                          iii vax

                                                                                                                                                          1

                                                                                                                                                          ii vxa

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iiii avvavvavx

                                                                                                                                                          1

                                                                                                                                                          1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                                          bull Spectral Representation

                                                                                                                                                          where

                                                                                                                                                          Check

                                                                                                                                                          bull Matrix Notation where ie

                                                                                                                                                          For the Basis Matrix

                                                                                                                                                          nvv 1

                                                                                                                                                          n

                                                                                                                                                          iii vax

                                                                                                                                                          1

                                                                                                                                                          ii vxa

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iiii avvavvavx

                                                                                                                                                          1

                                                                                                                                                          1

                                                                                                                                                          aBx Bxa tt xBa t

                                                                                                                                                          nvvB 1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Orthonormal Basis (cont)

                                                                                                                                                          bull Spectral Representation

                                                                                                                                                          where

                                                                                                                                                          Check

                                                                                                                                                          bull Matrix Notation where ie

                                                                                                                                                          is called transform of

                                                                                                                                                          (eg Fourier or Wavelet)

                                                                                                                                                          nvv 1

                                                                                                                                                          n

                                                                                                                                                          iii vax

                                                                                                                                                          1

                                                                                                                                                          ii vxa

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iii

                                                                                                                                                          n

                                                                                                                                                          iiii avvavvavx

                                                                                                                                                          1

                                                                                                                                                          1

                                                                                                                                                          aBx Bxa tt xBa t

                                                                                                                                                          xa

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Parseval identity for

                                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                                          x

                                                                                                                                                          nvv 1

                                                                                                                                                          2

                                                                                                                                                          1

                                                                                                                                                          22

                                                                                                                                                          1

                                                                                                                                                          2 aavxx

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Parseval identity for

                                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                                          bull Pythagorean theorem

                                                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                                                          bull ANOVA - sums of squares

                                                                                                                                                          x

                                                                                                                                                          nvv 1

                                                                                                                                                          2

                                                                                                                                                          1

                                                                                                                                                          22

                                                                                                                                                          1

                                                                                                                                                          2 aavxx

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Parseval identity for

                                                                                                                                                          in subsp genrsquod by o n basis

                                                                                                                                                          bull Pythagorean theorem

                                                                                                                                                          bull ldquoDecomposition of Energyrdquo

                                                                                                                                                          bull ANOVA - sums of squares

                                                                                                                                                          bull Transform has same length as

                                                                                                                                                          ie ldquorotation in rdquo

                                                                                                                                                          x

                                                                                                                                                          nvv 1

                                                                                                                                                          2

                                                                                                                                                          1

                                                                                                                                                          22

                                                                                                                                                          1

                                                                                                                                                          2 aavxx

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          a xd

                                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                                          xV

                                                                                                                                                          V

                                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                          bull Find that Solves

                                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                                          xV

                                                                                                                                                          V

                                                                                                                                                          VxPV vxVv

                                                                                                                                                          min

                                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                          bull Find that Solves

                                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                                          bull For Inner Product (Hilbert) Space

                                                                                                                                                          Exists and is Unique

                                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                                          xV

                                                                                                                                                          V

                                                                                                                                                          VxPV vxVv

                                                                                                                                                          min

                                                                                                                                                          xPV

                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          d VB

                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                          tVVV

                                                                                                                                                          1

                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          d VB

                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                          tVVV

                                                                                                                                                          1

                                                                                                                                                          tVV

                                                                                                                                                          tVVV BBBBP

                                                                                                                                                          1

                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                                          (note same operation underlies least squares)

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          d VB

                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                          tVVV

                                                                                                                                                          1

                                                                                                                                                          tVV

                                                                                                                                                          tVVV BBBBP

                                                                                                                                                          1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                                          nnVtV IBB

                                                                                                                                                          10

                                                                                                                                                          01

                                                                                                                                                          1

                                                                                                                                                          111

                                                                                                                                                          1

                                                                                                                                                          1

                                                                                                                                                          nnn

                                                                                                                                                          n

                                                                                                                                                          ntn

                                                                                                                                                          t

                                                                                                                                                          vvvv

                                                                                                                                                          vvvv

                                                                                                                                                          vv

                                                                                                                                                          v

                                                                                                                                                          v

                                                                                                                                                          nvv 1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                                          bull So =

                                                                                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                          (Recall Right Multrsquon)

                                                                                                                                                          nnVtV IBB

                                                                                                                                                          10

                                                                                                                                                          01

                                                                                                                                                          1

                                                                                                                                                          111

                                                                                                                                                          1

                                                                                                                                                          1

                                                                                                                                                          nnn

                                                                                                                                                          n

                                                                                                                                                          ntn

                                                                                                                                                          t

                                                                                                                                                          vvvv

                                                                                                                                                          vvvv

                                                                                                                                                          vv

                                                                                                                                                          v

                                                                                                                                                          v

                                                                                                                                                          xBBxP tVVV

                                                                                                                                                          x V

                                                                                                                                                          nvv 1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                                          and

                                                                                                                                                          V

                                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                                          and

                                                                                                                                                          bull Parseval Inequality

                                                                                                                                                          V

                                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                                          2

                                                                                                                                                          1

                                                                                                                                                          22

                                                                                                                                                          1

                                                                                                                                                          22 aavxxxP

                                                                                                                                                          n

                                                                                                                                                          ii

                                                                                                                                                          n

                                                                                                                                                          iiV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                          ddU IUU t

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                          ddU IUU t

                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                          i ii 2

                                                                                                                                                          1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                          ddU IUU t

                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                          i ii 2

                                                                                                                                                          1

                                                                                                                                                          U

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                          For a Matrix

                                                                                                                                                          Find

                                                                                                                                                          ndX

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                          For a Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          with Entries

                                                                                                                                                          called Singular Values

                                                                                                                                                          ndX

                                                                                                                                                          ndS

                                                                                                                                                          )min(1 ndss

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                          For a Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          with Entries

                                                                                                                                                          called Singular Values

                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                          (recall )

                                                                                                                                                          ndX

                                                                                                                                                          ndS

                                                                                                                                                          )min(1 ndss

                                                                                                                                                          ddU nnV

                                                                                                                                                          IVVUU tt

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                          For a Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          with Entries

                                                                                                                                                          called Singular Values

                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                          (recall )

                                                                                                                                                          So That

                                                                                                                                                          ndX

                                                                                                                                                          ndS

                                                                                                                                                          )min(1 ndss

                                                                                                                                                          ddU nnV

                                                                                                                                                          IVVUU tt tUSVX

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                          bull First rotate

                                                                                                                                                          X

                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                          bull First rotate

                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                          X

                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                          is

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                          bull First rotate

                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                          bull Third rotate again

                                                                                                                                                          X

                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                          is

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                          bull First rotate

                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                          bull Third rotate again

                                                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                                                          X

                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                          is

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          Useful Labeling

                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          )min(1 dnss

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          Useful Labeling

                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          )min(1 dnss

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          r

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          Useful Labeling

                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                          Let = of positive singular values

                                                                                                                                                          )min(1 dnss

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          r

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          Useful Labeling

                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                          Let = of positive singular values

                                                                                                                                                          Then

                                                                                                                                                          Where are truncations of

                                                                                                                                                          trnrrrd VSUX

                                                                                                                                                          VSU

                                                                                                                                                          )min(1 dnss

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Full Representation

                                                                                                                                                          =

                                                                                                                                                          Graphics Display Assumes

                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                          tV

                                                                                                                                                          nd

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Full Representation

                                                                                                                                                          =

                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                          tV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Full Representation

                                                                                                                                                          =

                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                          All 0s in Bottom

                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                          tV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                          =

                                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                                          ndX ddU nnS nn

                                                                                                                                                          tV

                                                                                                                                                          nnd 0

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                                          tV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                          =

                                                                                                                                                          Also Some of These May be 0

                                                                                                                                                          ndX ndU nnS nn

                                                                                                                                                          tV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          = ndX rdU

                                                                                                                                                          rrS nrtV

                                                                                                                                                          0

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          =

                                                                                                                                                          These Get 0ed Out

                                                                                                                                                          ndX rdU

                                                                                                                                                          rrS nrtV

                                                                                                                                                          0

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          SVD Compact Representation

                                                                                                                                                          = ndX rdU

                                                                                                                                                          rrS nrtV

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                          Find

                                                                                                                                                          ddX

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          Called Eigenvalues

                                                                                                                                                          Convenient Ordering

                                                                                                                                                          ddX

                                                                                                                                                          d

                                                                                                                                                          D

                                                                                                                                                          0

                                                                                                                                                          01

                                                                                                                                                          n 1

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                          (ie )

                                                                                                                                                          ddX

                                                                                                                                                          d

                                                                                                                                                          D

                                                                                                                                                          0

                                                                                                                                                          01

                                                                                                                                                          ddB

                                                                                                                                                          ddtt IBBBB

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                          (ie )

                                                                                                                                                          So that ie

                                                                                                                                                          ddX

                                                                                                                                                          d

                                                                                                                                                          D

                                                                                                                                                          0

                                                                                                                                                          01

                                                                                                                                                          ddB

                                                                                                                                                          ddtt IBBBB

                                                                                                                                                          DBBX tBDBX

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                          (looks similar)

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                          Complex (uses )

                                                                                                                                                          VU

                                                                                                                                                          1i

                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                          VU

                                                                                                                                                          X

                                                                                                                                                          BVU

                                                                                                                                                          1i

                                                                                                                                                          • Participant Presentations
                                                                                                                                                          • PCA to find clusters
                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                          • SiZer Background
                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                          • SiZer Overview
                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                          • New Subpopulation View
                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                                            bull Spectral Representation

                                                                                                                                                            where

                                                                                                                                                            Check

                                                                                                                                                            nvv 1

                                                                                                                                                            n

                                                                                                                                                            iii vax

                                                                                                                                                            1

                                                                                                                                                            ii vxa

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iiii avvavvavx

                                                                                                                                                            1

                                                                                                                                                            1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                                            bull Spectral Representation

                                                                                                                                                            where

                                                                                                                                                            Check

                                                                                                                                                            bull Matrix Notation where ie

                                                                                                                                                            For the Basis Matrix

                                                                                                                                                            nvv 1

                                                                                                                                                            n

                                                                                                                                                            iii vax

                                                                                                                                                            1

                                                                                                                                                            ii vxa

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iiii avvavvavx

                                                                                                                                                            1

                                                                                                                                                            1

                                                                                                                                                            aBx Bxa tt xBa t

                                                                                                                                                            nvvB 1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Orthonormal Basis (cont)

                                                                                                                                                            bull Spectral Representation

                                                                                                                                                            where

                                                                                                                                                            Check

                                                                                                                                                            bull Matrix Notation where ie

                                                                                                                                                            is called transform of

                                                                                                                                                            (eg Fourier or Wavelet)

                                                                                                                                                            nvv 1

                                                                                                                                                            n

                                                                                                                                                            iii vax

                                                                                                                                                            1

                                                                                                                                                            ii vxa

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iii

                                                                                                                                                            n

                                                                                                                                                            iiii avvavvavx

                                                                                                                                                            1

                                                                                                                                                            1

                                                                                                                                                            aBx Bxa tt xBa t

                                                                                                                                                            xa

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Parseval identity for

                                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                                            x

                                                                                                                                                            nvv 1

                                                                                                                                                            2

                                                                                                                                                            1

                                                                                                                                                            22

                                                                                                                                                            1

                                                                                                                                                            2 aavxx

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Parseval identity for

                                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                                            bull Pythagorean theorem

                                                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                                                            bull ANOVA - sums of squares

                                                                                                                                                            x

                                                                                                                                                            nvv 1

                                                                                                                                                            2

                                                                                                                                                            1

                                                                                                                                                            22

                                                                                                                                                            1

                                                                                                                                                            2 aavxx

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Parseval identity for

                                                                                                                                                            in subsp genrsquod by o n basis

                                                                                                                                                            bull Pythagorean theorem

                                                                                                                                                            bull ldquoDecomposition of Energyrdquo

                                                                                                                                                            bull ANOVA - sums of squares

                                                                                                                                                            bull Transform has same length as

                                                                                                                                                            ie ldquorotation in rdquo

                                                                                                                                                            x

                                                                                                                                                            nvv 1

                                                                                                                                                            2

                                                                                                                                                            1

                                                                                                                                                            22

                                                                                                                                                            1

                                                                                                                                                            2 aavxx

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            a xd

                                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                                            xV

                                                                                                                                                            V

                                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                            bull Find that Solves

                                                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                                            xV

                                                                                                                                                            V

                                                                                                                                                            VxPV vxVv

                                                                                                                                                            min

                                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                            bull Find that Solves

                                                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                                                            bull For Inner Product (Hilbert) Space

                                                                                                                                                            Exists and is Unique

                                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                                            xV

                                                                                                                                                            V

                                                                                                                                                            VxPV vxVv

                                                                                                                                                            min

                                                                                                                                                            xPV

                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            d VB

                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                            tVVV

                                                                                                                                                            1

                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            d VB

                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                            tVVV

                                                                                                                                                            1

                                                                                                                                                            tVV

                                                                                                                                                            tVVV BBBBP

                                                                                                                                                            1

                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                                            (note same operation underlies least squares)

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            d VB

                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                            tVVV

                                                                                                                                                            1

                                                                                                                                                            tVV

                                                                                                                                                            tVVV BBBBP

                                                                                                                                                            1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                                            nnVtV IBB

                                                                                                                                                            10

                                                                                                                                                            01

                                                                                                                                                            1

                                                                                                                                                            111

                                                                                                                                                            1

                                                                                                                                                            1

                                                                                                                                                            nnn

                                                                                                                                                            n

                                                                                                                                                            ntn

                                                                                                                                                            t

                                                                                                                                                            vvvv

                                                                                                                                                            vvvv

                                                                                                                                                            vv

                                                                                                                                                            v

                                                                                                                                                            v

                                                                                                                                                            nvv 1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                                            bull So =

                                                                                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                            (Recall Right Multrsquon)

                                                                                                                                                            nnVtV IBB

                                                                                                                                                            10

                                                                                                                                                            01

                                                                                                                                                            1

                                                                                                                                                            111

                                                                                                                                                            1

                                                                                                                                                            1

                                                                                                                                                            nnn

                                                                                                                                                            n

                                                                                                                                                            ntn

                                                                                                                                                            t

                                                                                                                                                            vvvv

                                                                                                                                                            vvvv

                                                                                                                                                            vv

                                                                                                                                                            v

                                                                                                                                                            v

                                                                                                                                                            xBBxP tVVV

                                                                                                                                                            x V

                                                                                                                                                            nvv 1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                                            and

                                                                                                                                                            V

                                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                                            and

                                                                                                                                                            bull Parseval Inequality

                                                                                                                                                            V

                                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                                            2

                                                                                                                                                            1

                                                                                                                                                            22

                                                                                                                                                            1

                                                                                                                                                            22 aavxxxP

                                                                                                                                                            n

                                                                                                                                                            ii

                                                                                                                                                            n

                                                                                                                                                            iiV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                            ddU IUU t

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                            ddU IUU t

                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                            i ii 2

                                                                                                                                                            1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                            ddU IUU t

                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                            i ii 2

                                                                                                                                                            1

                                                                                                                                                            U

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                            For a Matrix

                                                                                                                                                            Find

                                                                                                                                                            ndX

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                            For a Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            with Entries

                                                                                                                                                            called Singular Values

                                                                                                                                                            ndX

                                                                                                                                                            ndS

                                                                                                                                                            )min(1 ndss

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                            For a Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            with Entries

                                                                                                                                                            called Singular Values

                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                            (recall )

                                                                                                                                                            ndX

                                                                                                                                                            ndS

                                                                                                                                                            )min(1 ndss

                                                                                                                                                            ddU nnV

                                                                                                                                                            IVVUU tt

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                            For a Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            with Entries

                                                                                                                                                            called Singular Values

                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                            (recall )

                                                                                                                                                            So That

                                                                                                                                                            ndX

                                                                                                                                                            ndS

                                                                                                                                                            )min(1 ndss

                                                                                                                                                            ddU nnV

                                                                                                                                                            IVVUU tt tUSVX

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                            bull First rotate

                                                                                                                                                            X

                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                            bull First rotate

                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                            X

                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                            is

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                            bull First rotate

                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                            bull Third rotate again

                                                                                                                                                            X

                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                            is

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                            bull First rotate

                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                            bull Third rotate again

                                                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                                                            X

                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                            is

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            Useful Labeling

                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            )min(1 dnss

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            Useful Labeling

                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            )min(1 dnss

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            r

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            Useful Labeling

                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                            Let = of positive singular values

                                                                                                                                                            )min(1 dnss

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            r

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            Useful Labeling

                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                            Let = of positive singular values

                                                                                                                                                            Then

                                                                                                                                                            Where are truncations of

                                                                                                                                                            trnrrrd VSUX

                                                                                                                                                            VSU

                                                                                                                                                            )min(1 dnss

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Full Representation

                                                                                                                                                            =

                                                                                                                                                            Graphics Display Assumes

                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                            tV

                                                                                                                                                            nd

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Full Representation

                                                                                                                                                            =

                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                            tV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Full Representation

                                                                                                                                                            =

                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                            All 0s in Bottom

                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                            tV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                            =

                                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                                            ndX ddU nnS nn

                                                                                                                                                            tV

                                                                                                                                                            nnd 0

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                                            tV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                            =

                                                                                                                                                            Also Some of These May be 0

                                                                                                                                                            ndX ndU nnS nn

                                                                                                                                                            tV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            = ndX rdU

                                                                                                                                                            rrS nrtV

                                                                                                                                                            0

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            =

                                                                                                                                                            These Get 0ed Out

                                                                                                                                                            ndX rdU

                                                                                                                                                            rrS nrtV

                                                                                                                                                            0

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            SVD Compact Representation

                                                                                                                                                            = ndX rdU

                                                                                                                                                            rrS nrtV

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                            Find

                                                                                                                                                            ddX

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            Called Eigenvalues

                                                                                                                                                            Convenient Ordering

                                                                                                                                                            ddX

                                                                                                                                                            d

                                                                                                                                                            D

                                                                                                                                                            0

                                                                                                                                                            01

                                                                                                                                                            n 1

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                            (ie )

                                                                                                                                                            ddX

                                                                                                                                                            d

                                                                                                                                                            D

                                                                                                                                                            0

                                                                                                                                                            01

                                                                                                                                                            ddB

                                                                                                                                                            ddtt IBBBB

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                            (ie )

                                                                                                                                                            So that ie

                                                                                                                                                            ddX

                                                                                                                                                            d

                                                                                                                                                            D

                                                                                                                                                            0

                                                                                                                                                            01

                                                                                                                                                            ddB

                                                                                                                                                            ddtt IBBBB

                                                                                                                                                            DBBX tBDBX

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                            (looks similar)

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                            Complex (uses )

                                                                                                                                                            VU

                                                                                                                                                            1i

                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                            VU

                                                                                                                                                            X

                                                                                                                                                            BVU

                                                                                                                                                            1i

                                                                                                                                                            • Participant Presentations
                                                                                                                                                            • PCA to find clusters
                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                            • SiZer Background
                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                            • SiZer Overview
                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                            • New Subpopulation View
                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                                              bull Spectral Representation

                                                                                                                                                              where

                                                                                                                                                              Check

                                                                                                                                                              bull Matrix Notation where ie

                                                                                                                                                              For the Basis Matrix

                                                                                                                                                              nvv 1

                                                                                                                                                              n

                                                                                                                                                              iii vax

                                                                                                                                                              1

                                                                                                                                                              ii vxa

                                                                                                                                                              iii

                                                                                                                                                              n

                                                                                                                                                              iii

                                                                                                                                                              n

                                                                                                                                                              iiii avvavvavx

                                                                                                                                                              1

                                                                                                                                                              1

                                                                                                                                                              aBx Bxa tt xBa t

                                                                                                                                                              nvvB 1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Orthonormal Basis (cont)

                                                                                                                                                              bull Spectral Representation

                                                                                                                                                              where

                                                                                                                                                              Check

                                                                                                                                                              bull Matrix Notation where ie

                                                                                                                                                              is called transform of

                                                                                                                                                              (eg Fourier or Wavelet)

                                                                                                                                                              nvv 1

                                                                                                                                                              n

                                                                                                                                                              iii vax

                                                                                                                                                              1

                                                                                                                                                              ii vxa

                                                                                                                                                              iii

                                                                                                                                                              n

                                                                                                                                                              iii

                                                                                                                                                              n

                                                                                                                                                              iiii avvavvavx

                                                                                                                                                              1

                                                                                                                                                              1

                                                                                                                                                              aBx Bxa tt xBa t

                                                                                                                                                              xa

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Parseval identity for

                                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                                              x

                                                                                                                                                              nvv 1

                                                                                                                                                              2

                                                                                                                                                              1

                                                                                                                                                              22

                                                                                                                                                              1

                                                                                                                                                              2 aavxx

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Parseval identity for

                                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                                              bull Pythagorean theorem

                                                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                                                              bull ANOVA - sums of squares

                                                                                                                                                              x

                                                                                                                                                              nvv 1

                                                                                                                                                              2

                                                                                                                                                              1

                                                                                                                                                              22

                                                                                                                                                              1

                                                                                                                                                              2 aavxx

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Parseval identity for

                                                                                                                                                              in subsp genrsquod by o n basis

                                                                                                                                                              bull Pythagorean theorem

                                                                                                                                                              bull ldquoDecomposition of Energyrdquo

                                                                                                                                                              bull ANOVA - sums of squares

                                                                                                                                                              bull Transform has same length as

                                                                                                                                                              ie ldquorotation in rdquo

                                                                                                                                                              x

                                                                                                                                                              nvv 1

                                                                                                                                                              2

                                                                                                                                                              1

                                                                                                                                                              22

                                                                                                                                                              1

                                                                                                                                                              2 aavxx

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              a xd

                                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                                              xV

                                                                                                                                                              V

                                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                              bull Find that Solves

                                                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                                              xV

                                                                                                                                                              V

                                                                                                                                                              VxPV vxVv

                                                                                                                                                              min

                                                                                                                                                              Projection of a Vector onto a Subspace

                                                                                                                                                              bull Idea Member of that is Closest to

                                                                                                                                                              (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                              bull Find that Solves

                                                                                                                                                              (ldquoLeast Squaresrdquo)

                                                                                                                                                              bull For Inner Product (Hilbert) Space

                                                                                                                                                              Exists and is Unique

                                                                                                                                                              Review of Linear Algebra (Cont)x

                                                                                                                                                              xV

                                                                                                                                                              V

                                                                                                                                                              VxPV vxVv

                                                                                                                                                              min

                                                                                                                                                              xPV

                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              d VB

                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                              tVVV

                                                                                                                                                              1

                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              d VB

                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                              tVVV

                                                                                                                                                              1

                                                                                                                                                              tVV

                                                                                                                                                              tVVV BBBBP

                                                                                                                                                              1

                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                                              (note same operation underlies least squares)

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              d VB

                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                              tVVV

                                                                                                                                                              1

                                                                                                                                                              tVV

                                                                                                                                                              tVVV BBBBP

                                                                                                                                                              1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                                              nnVtV IBB

                                                                                                                                                              10

                                                                                                                                                              01

                                                                                                                                                              1

                                                                                                                                                              111

                                                                                                                                                              1

                                                                                                                                                              1

                                                                                                                                                              nnn

                                                                                                                                                              n

                                                                                                                                                              ntn

                                                                                                                                                              t

                                                                                                                                                              vvvv

                                                                                                                                                              vvvv

                                                                                                                                                              vv

                                                                                                                                                              v

                                                                                                                                                              v

                                                                                                                                                              nvv 1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                                              bull So =

                                                                                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                              (Recall Right Multrsquon)

                                                                                                                                                              nnVtV IBB

                                                                                                                                                              10

                                                                                                                                                              01

                                                                                                                                                              1

                                                                                                                                                              111

                                                                                                                                                              1

                                                                                                                                                              1

                                                                                                                                                              nnn

                                                                                                                                                              n

                                                                                                                                                              ntn

                                                                                                                                                              t

                                                                                                                                                              vvvv

                                                                                                                                                              vvvv

                                                                                                                                                              vv

                                                                                                                                                              v

                                                                                                                                                              v

                                                                                                                                                              xBBxP tVVV

                                                                                                                                                              x V

                                                                                                                                                              nvv 1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                                              and

                                                                                                                                                              V

                                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                                              and

                                                                                                                                                              bull Parseval Inequality

                                                                                                                                                              V

                                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                                              2

                                                                                                                                                              1

                                                                                                                                                              22

                                                                                                                                                              1

                                                                                                                                                              22 aavxxxP

                                                                                                                                                              n

                                                                                                                                                              ii

                                                                                                                                                              n

                                                                                                                                                              iiV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                              ddU IUU t

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                              ddU IUU t

                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                              i ii 2

                                                                                                                                                              1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                              ddU IUU t

                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                              i ii 2

                                                                                                                                                              1

                                                                                                                                                              U

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                              For a Matrix

                                                                                                                                                              Find

                                                                                                                                                              ndX

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                              For a Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              with Entries

                                                                                                                                                              called Singular Values

                                                                                                                                                              ndX

                                                                                                                                                              ndS

                                                                                                                                                              )min(1 ndss

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                              For a Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              with Entries

                                                                                                                                                              called Singular Values

                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                              (recall )

                                                                                                                                                              ndX

                                                                                                                                                              ndS

                                                                                                                                                              )min(1 ndss

                                                                                                                                                              ddU nnV

                                                                                                                                                              IVVUU tt

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                              For a Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              with Entries

                                                                                                                                                              called Singular Values

                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                              (recall )

                                                                                                                                                              So That

                                                                                                                                                              ndX

                                                                                                                                                              ndS

                                                                                                                                                              )min(1 ndss

                                                                                                                                                              ddU nnV

                                                                                                                                                              IVVUU tt tUSVX

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                              bull First rotate

                                                                                                                                                              X

                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                              bull First rotate

                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                              X

                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                              is

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                              bull First rotate

                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                              bull Third rotate again

                                                                                                                                                              X

                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                              is

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                              bull First rotate

                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                              bull Third rotate again

                                                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                                                              X

                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                              is

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              Useful Labeling

                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              )min(1 dnss

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              Useful Labeling

                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              )min(1 dnss

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              r

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              Useful Labeling

                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                              Let = of positive singular values

                                                                                                                                                              )min(1 dnss

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              r

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              Useful Labeling

                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                              Let = of positive singular values

                                                                                                                                                              Then

                                                                                                                                                              Where are truncations of

                                                                                                                                                              trnrrrd VSUX

                                                                                                                                                              VSU

                                                                                                                                                              )min(1 dnss

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Full Representation

                                                                                                                                                              =

                                                                                                                                                              Graphics Display Assumes

                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                              tV

                                                                                                                                                              nd

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Full Representation

                                                                                                                                                              =

                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                              tV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Full Representation

                                                                                                                                                              =

                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                              All 0s in Bottom

                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                              tV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                              =

                                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                                              ndX ddU nnS nn

                                                                                                                                                              tV

                                                                                                                                                              nnd 0

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                                              tV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                              =

                                                                                                                                                              Also Some of These May be 0

                                                                                                                                                              ndX ndU nnS nn

                                                                                                                                                              tV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              = ndX rdU

                                                                                                                                                              rrS nrtV

                                                                                                                                                              0

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              =

                                                                                                                                                              These Get 0ed Out

                                                                                                                                                              ndX rdU

                                                                                                                                                              rrS nrtV

                                                                                                                                                              0

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              SVD Compact Representation

                                                                                                                                                              = ndX rdU

                                                                                                                                                              rrS nrtV

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                              Find

                                                                                                                                                              ddX

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              Called Eigenvalues

                                                                                                                                                              Convenient Ordering

                                                                                                                                                              ddX

                                                                                                                                                              d

                                                                                                                                                              D

                                                                                                                                                              0

                                                                                                                                                              01

                                                                                                                                                              n 1

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                              (ie )

                                                                                                                                                              ddX

                                                                                                                                                              d

                                                                                                                                                              D

                                                                                                                                                              0

                                                                                                                                                              01

                                                                                                                                                              ddB

                                                                                                                                                              ddtt IBBBB

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                              (ie )

                                                                                                                                                              So that ie

                                                                                                                                                              ddX

                                                                                                                                                              d

                                                                                                                                                              D

                                                                                                                                                              0

                                                                                                                                                              01

                                                                                                                                                              ddB

                                                                                                                                                              ddtt IBBBB

                                                                                                                                                              DBBX tBDBX

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                              (looks similar)

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                              Complex (uses )

                                                                                                                                                              VU

                                                                                                                                                              1i

                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                              VU

                                                                                                                                                              X

                                                                                                                                                              BVU

                                                                                                                                                              1i

                                                                                                                                                              • Participant Presentations
                                                                                                                                                              • PCA to find clusters
                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                              • SiZer Background
                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                              • SiZer Overview
                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                              • New Subpopulation View
                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Orthonormal Basis (cont)

                                                                                                                                                                bull Spectral Representation

                                                                                                                                                                where

                                                                                                                                                                Check

                                                                                                                                                                bull Matrix Notation where ie

                                                                                                                                                                is called transform of

                                                                                                                                                                (eg Fourier or Wavelet)

                                                                                                                                                                nvv 1

                                                                                                                                                                n

                                                                                                                                                                iii vax

                                                                                                                                                                1

                                                                                                                                                                ii vxa

                                                                                                                                                                iii

                                                                                                                                                                n

                                                                                                                                                                iii

                                                                                                                                                                n

                                                                                                                                                                iiii avvavvavx

                                                                                                                                                                1

                                                                                                                                                                1

                                                                                                                                                                aBx Bxa tt xBa t

                                                                                                                                                                xa

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Parseval identity for

                                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                                x

                                                                                                                                                                nvv 1

                                                                                                                                                                2

                                                                                                                                                                1

                                                                                                                                                                22

                                                                                                                                                                1

                                                                                                                                                                2 aavxx

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Parseval identity for

                                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                                bull Pythagorean theorem

                                                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                                                x

                                                                                                                                                                nvv 1

                                                                                                                                                                2

                                                                                                                                                                1

                                                                                                                                                                22

                                                                                                                                                                1

                                                                                                                                                                2 aavxx

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Parseval identity for

                                                                                                                                                                in subsp genrsquod by o n basis

                                                                                                                                                                bull Pythagorean theorem

                                                                                                                                                                bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                bull ANOVA - sums of squares

                                                                                                                                                                bull Transform has same length as

                                                                                                                                                                ie ldquorotation in rdquo

                                                                                                                                                                x

                                                                                                                                                                nvv 1

                                                                                                                                                                2

                                                                                                                                                                1

                                                                                                                                                                22

                                                                                                                                                                1

                                                                                                                                                                2 aavxx

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                a xd

                                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                                xV

                                                                                                                                                                V

                                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                bull Find that Solves

                                                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                                xV

                                                                                                                                                                V

                                                                                                                                                                VxPV vxVv

                                                                                                                                                                min

                                                                                                                                                                Projection of a Vector onto a Subspace

                                                                                                                                                                bull Idea Member of that is Closest to

                                                                                                                                                                (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                bull Find that Solves

                                                                                                                                                                (ldquoLeast Squaresrdquo)

                                                                                                                                                                bull For Inner Product (Hilbert) Space

                                                                                                                                                                Exists and is Unique

                                                                                                                                                                Review of Linear Algebra (Cont)x

                                                                                                                                                                xV

                                                                                                                                                                V

                                                                                                                                                                VxPV vxVv

                                                                                                                                                                min

                                                                                                                                                                xPV

                                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                d VB

                                                                                                                                                                xBBBBxP tVV

                                                                                                                                                                tVVV

                                                                                                                                                                1

                                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                d VB

                                                                                                                                                                xBBBBxP tVV

                                                                                                                                                                tVVV

                                                                                                                                                                1

                                                                                                                                                                tVV

                                                                                                                                                                tVVV BBBBP

                                                                                                                                                                1

                                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                                (note same operation underlies least squares)

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                d VB

                                                                                                                                                                xBBBBxP tVV

                                                                                                                                                                tVVV

                                                                                                                                                                1

                                                                                                                                                                tVV

                                                                                                                                                                tVVV BBBBP

                                                                                                                                                                1

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                                nnVtV IBB

                                                                                                                                                                10

                                                                                                                                                                01

                                                                                                                                                                1

                                                                                                                                                                111

                                                                                                                                                                1

                                                                                                                                                                1

                                                                                                                                                                nnn

                                                                                                                                                                n

                                                                                                                                                                ntn

                                                                                                                                                                t

                                                                                                                                                                vvvv

                                                                                                                                                                vvvv

                                                                                                                                                                vv

                                                                                                                                                                v

                                                                                                                                                                v

                                                                                                                                                                nvv 1

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                                bull So =

                                                                                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                (Recall Right Multrsquon)

                                                                                                                                                                nnVtV IBB

                                                                                                                                                                10

                                                                                                                                                                01

                                                                                                                                                                1

                                                                                                                                                                111

                                                                                                                                                                1

                                                                                                                                                                1

                                                                                                                                                                nnn

                                                                                                                                                                n

                                                                                                                                                                ntn

                                                                                                                                                                t

                                                                                                                                                                vvvv

                                                                                                                                                                vvvv

                                                                                                                                                                vv

                                                                                                                                                                v

                                                                                                                                                                v

                                                                                                                                                                xBBxP tVVV

                                                                                                                                                                x V

                                                                                                                                                                nvv 1

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                                and

                                                                                                                                                                V

                                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                                and

                                                                                                                                                                bull Parseval Inequality

                                                                                                                                                                V

                                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                                2

                                                                                                                                                                1

                                                                                                                                                                22

                                                                                                                                                                1

                                                                                                                                                                22 aavxxxP

                                                                                                                                                                n

                                                                                                                                                                ii

                                                                                                                                                                n

                                                                                                                                                                iiV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                ddU IUU t

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                ddU IUU t

                                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                                i ii 2

                                                                                                                                                                1

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                ddU IUU t

                                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                                i ii 2

                                                                                                                                                                1

                                                                                                                                                                U

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                For a Matrix

                                                                                                                                                                Find

                                                                                                                                                                ndX

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                For a Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                with Entries

                                                                                                                                                                called Singular Values

                                                                                                                                                                ndX

                                                                                                                                                                ndS

                                                                                                                                                                )min(1 ndss

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                For a Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                with Entries

                                                                                                                                                                called Singular Values

                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                (recall )

                                                                                                                                                                ndX

                                                                                                                                                                ndS

                                                                                                                                                                )min(1 ndss

                                                                                                                                                                ddU nnV

                                                                                                                                                                IVVUU tt

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                For a Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                with Entries

                                                                                                                                                                called Singular Values

                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                (recall )

                                                                                                                                                                So That

                                                                                                                                                                ndX

                                                                                                                                                                ndS

                                                                                                                                                                )min(1 ndss

                                                                                                                                                                ddU nnV

                                                                                                                                                                IVVUU tt tUSVX

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                bull First rotate

                                                                                                                                                                X

                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                bull First rotate

                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                X

                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                is

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                bull First rotate

                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                bull Third rotate again

                                                                                                                                                                X

                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                is

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                bull First rotate

                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                bull Third rotate again

                                                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                                                X

                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                is

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                Useful Labeling

                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                )min(1 dnss

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                Useful Labeling

                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                )min(1 dnss

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                r

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                Useful Labeling

                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                )min(1 dnss

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                r

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                Useful Labeling

                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                Then

                                                                                                                                                                Where are truncations of

                                                                                                                                                                trnrrrd VSUX

                                                                                                                                                                VSU

                                                                                                                                                                )min(1 dnss

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Full Representation

                                                                                                                                                                =

                                                                                                                                                                Graphics Display Assumes

                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                tV

                                                                                                                                                                nd

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Full Representation

                                                                                                                                                                =

                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                tV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Full Representation

                                                                                                                                                                =

                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                All 0s in Bottom

                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                tV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                =

                                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                                tV

                                                                                                                                                                nnd 0

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                                tV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                =

                                                                                                                                                                Also Some of These May be 0

                                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                                tV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                = ndX rdU

                                                                                                                                                                rrS nrtV

                                                                                                                                                                0

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                =

                                                                                                                                                                These Get 0ed Out

                                                                                                                                                                ndX rdU

                                                                                                                                                                rrS nrtV

                                                                                                                                                                0

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                = ndX rdU

                                                                                                                                                                rrS nrtV

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                Find

                                                                                                                                                                ddX

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                Convenient Ordering

                                                                                                                                                                ddX

                                                                                                                                                                d

                                                                                                                                                                D

                                                                                                                                                                0

                                                                                                                                                                01

                                                                                                                                                                n 1

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                (ie )

                                                                                                                                                                ddX

                                                                                                                                                                d

                                                                                                                                                                D

                                                                                                                                                                0

                                                                                                                                                                01

                                                                                                                                                                ddB

                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                (ie )

                                                                                                                                                                So that ie

                                                                                                                                                                ddX

                                                                                                                                                                d

                                                                                                                                                                D

                                                                                                                                                                0

                                                                                                                                                                01

                                                                                                                                                                ddB

                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                (looks similar)

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                Complex (uses )

                                                                                                                                                                VU

                                                                                                                                                                1i

                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                VU

                                                                                                                                                                X

                                                                                                                                                                BVU

                                                                                                                                                                1i

                                                                                                                                                                • Participant Presentations
                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                • SiZer Background
                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                • SiZer Overview
                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Parseval identity for

                                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                                  x

                                                                                                                                                                  nvv 1

                                                                                                                                                                  2

                                                                                                                                                                  1

                                                                                                                                                                  22

                                                                                                                                                                  1

                                                                                                                                                                  2 aavxx

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Parseval identity for

                                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                                  bull Pythagorean theorem

                                                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                                                  x

                                                                                                                                                                  nvv 1

                                                                                                                                                                  2

                                                                                                                                                                  1

                                                                                                                                                                  22

                                                                                                                                                                  1

                                                                                                                                                                  2 aavxx

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Parseval identity for

                                                                                                                                                                  in subsp genrsquod by o n basis

                                                                                                                                                                  bull Pythagorean theorem

                                                                                                                                                                  bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                  bull ANOVA - sums of squares

                                                                                                                                                                  bull Transform has same length as

                                                                                                                                                                  ie ldquorotation in rdquo

                                                                                                                                                                  x

                                                                                                                                                                  nvv 1

                                                                                                                                                                  2

                                                                                                                                                                  1

                                                                                                                                                                  22

                                                                                                                                                                  1

                                                                                                                                                                  2 aavxx

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  a xd

                                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                                  xV

                                                                                                                                                                  V

                                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                  bull Find that Solves

                                                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                                  xV

                                                                                                                                                                  V

                                                                                                                                                                  VxPV vxVv

                                                                                                                                                                  min

                                                                                                                                                                  Projection of a Vector onto a Subspace

                                                                                                                                                                  bull Idea Member of that is Closest to

                                                                                                                                                                  (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                  bull Find that Solves

                                                                                                                                                                  (ldquoLeast Squaresrdquo)

                                                                                                                                                                  bull For Inner Product (Hilbert) Space

                                                                                                                                                                  Exists and is Unique

                                                                                                                                                                  Review of Linear Algebra (Cont)x

                                                                                                                                                                  xV

                                                                                                                                                                  V

                                                                                                                                                                  VxPV vxVv

                                                                                                                                                                  min

                                                                                                                                                                  xPV

                                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  d VB

                                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                                  tVVV

                                                                                                                                                                  1

                                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  d VB

                                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                                  tVVV

                                                                                                                                                                  1

                                                                                                                                                                  tVV

                                                                                                                                                                  tVVV BBBBP

                                                                                                                                                                  1

                                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                                                  (note same operation underlies least squares)

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  d VB

                                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                                  tVVV

                                                                                                                                                                  1

                                                                                                                                                                  tVV

                                                                                                                                                                  tVVV BBBBP

                                                                                                                                                                  1

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                                  nnVtV IBB

                                                                                                                                                                  10

                                                                                                                                                                  01

                                                                                                                                                                  1

                                                                                                                                                                  111

                                                                                                                                                                  1

                                                                                                                                                                  1

                                                                                                                                                                  nnn

                                                                                                                                                                  n

                                                                                                                                                                  ntn

                                                                                                                                                                  t

                                                                                                                                                                  vvvv

                                                                                                                                                                  vvvv

                                                                                                                                                                  vv

                                                                                                                                                                  v

                                                                                                                                                                  v

                                                                                                                                                                  nvv 1

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                                  bull So =

                                                                                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                  (Recall Right Multrsquon)

                                                                                                                                                                  nnVtV IBB

                                                                                                                                                                  10

                                                                                                                                                                  01

                                                                                                                                                                  1

                                                                                                                                                                  111

                                                                                                                                                                  1

                                                                                                                                                                  1

                                                                                                                                                                  nnn

                                                                                                                                                                  n

                                                                                                                                                                  ntn

                                                                                                                                                                  t

                                                                                                                                                                  vvvv

                                                                                                                                                                  vvvv

                                                                                                                                                                  vv

                                                                                                                                                                  v

                                                                                                                                                                  v

                                                                                                                                                                  xBBxP tVVV

                                                                                                                                                                  x V

                                                                                                                                                                  nvv 1

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                                  and

                                                                                                                                                                  V

                                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                                  and

                                                                                                                                                                  bull Parseval Inequality

                                                                                                                                                                  V

                                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                                  2

                                                                                                                                                                  1

                                                                                                                                                                  22

                                                                                                                                                                  1

                                                                                                                                                                  22 aavxxxP

                                                                                                                                                                  n

                                                                                                                                                                  ii

                                                                                                                                                                  n

                                                                                                                                                                  iiV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                  ddU IUU t

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                  ddU IUU t

                                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                                  i ii 2

                                                                                                                                                                  1

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                  ddU IUU t

                                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                                  i ii 2

                                                                                                                                                                  1

                                                                                                                                                                  U

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                  For a Matrix

                                                                                                                                                                  Find

                                                                                                                                                                  ndX

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                  For a Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  with Entries

                                                                                                                                                                  called Singular Values

                                                                                                                                                                  ndX

                                                                                                                                                                  ndS

                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                  For a Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  with Entries

                                                                                                                                                                  called Singular Values

                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                  (recall )

                                                                                                                                                                  ndX

                                                                                                                                                                  ndS

                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                  ddU nnV

                                                                                                                                                                  IVVUU tt

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                  For a Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  with Entries

                                                                                                                                                                  called Singular Values

                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                  (recall )

                                                                                                                                                                  So That

                                                                                                                                                                  ndX

                                                                                                                                                                  ndS

                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                  ddU nnV

                                                                                                                                                                  IVVUU tt tUSVX

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                  bull First rotate

                                                                                                                                                                  X

                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                  bull First rotate

                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                  X

                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                  is

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                  bull First rotate

                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                  X

                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                  is

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                  bull First rotate

                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                                                  X

                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                  is

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  Useful Labeling

                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  Useful Labeling

                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  r

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  Useful Labeling

                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  r

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  Useful Labeling

                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                  Then

                                                                                                                                                                  Where are truncations of

                                                                                                                                                                  trnrrrd VSUX

                                                                                                                                                                  VSU

                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                  =

                                                                                                                                                                  Graphics Display Assumes

                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                  tV

                                                                                                                                                                  nd

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                  =

                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                  tV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                  =

                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                  All 0s in Bottom

                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                  tV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                  =

                                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                                  tV

                                                                                                                                                                  nnd 0

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                                  tV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                  =

                                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                                  tV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  = ndX rdU

                                                                                                                                                                  rrS nrtV

                                                                                                                                                                  0

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  =

                                                                                                                                                                  These Get 0ed Out

                                                                                                                                                                  ndX rdU

                                                                                                                                                                  rrS nrtV

                                                                                                                                                                  0

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                  = ndX rdU

                                                                                                                                                                  rrS nrtV

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                  Find

                                                                                                                                                                  ddX

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  Called Eigenvalues

                                                                                                                                                                  Convenient Ordering

                                                                                                                                                                  ddX

                                                                                                                                                                  d

                                                                                                                                                                  D

                                                                                                                                                                  0

                                                                                                                                                                  01

                                                                                                                                                                  n 1

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                  (ie )

                                                                                                                                                                  ddX

                                                                                                                                                                  d

                                                                                                                                                                  D

                                                                                                                                                                  0

                                                                                                                                                                  01

                                                                                                                                                                  ddB

                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                  (ie )

                                                                                                                                                                  So that ie

                                                                                                                                                                  ddX

                                                                                                                                                                  d

                                                                                                                                                                  D

                                                                                                                                                                  0

                                                                                                                                                                  01

                                                                                                                                                                  ddB

                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                  (looks similar)

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                  Complex (uses )

                                                                                                                                                                  VU

                                                                                                                                                                  1i

                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                  VU

                                                                                                                                                                  X

                                                                                                                                                                  BVU

                                                                                                                                                                  1i

                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                  • SiZer Background
                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Parseval identity for

                                                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                                                    bull Pythagorean theorem

                                                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                                                    x

                                                                                                                                                                    nvv 1

                                                                                                                                                                    2

                                                                                                                                                                    1

                                                                                                                                                                    22

                                                                                                                                                                    1

                                                                                                                                                                    2 aavxx

                                                                                                                                                                    n

                                                                                                                                                                    ii

                                                                                                                                                                    n

                                                                                                                                                                    ii

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Parseval identity for

                                                                                                                                                                    in subsp genrsquod by o n basis

                                                                                                                                                                    bull Pythagorean theorem

                                                                                                                                                                    bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                    bull ANOVA - sums of squares

                                                                                                                                                                    bull Transform has same length as

                                                                                                                                                                    ie ldquorotation in rdquo

                                                                                                                                                                    x

                                                                                                                                                                    nvv 1

                                                                                                                                                                    2

                                                                                                                                                                    1

                                                                                                                                                                    22

                                                                                                                                                                    1

                                                                                                                                                                    2 aavxx

                                                                                                                                                                    n

                                                                                                                                                                    ii

                                                                                                                                                                    n

                                                                                                                                                                    ii

                                                                                                                                                                    a xd

                                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                                    xV

                                                                                                                                                                    V

                                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                    bull Find that Solves

                                                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                                    xV

                                                                                                                                                                    V

                                                                                                                                                                    VxPV vxVv

                                                                                                                                                                    min

                                                                                                                                                                    Projection of a Vector onto a Subspace

                                                                                                                                                                    bull Idea Member of that is Closest to

                                                                                                                                                                    (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                    bull Find that Solves

                                                                                                                                                                    (ldquoLeast Squaresrdquo)

                                                                                                                                                                    bull For Inner Product (Hilbert) Space

                                                                                                                                                                    Exists and is Unique

                                                                                                                                                                    Review of Linear Algebra (Cont)x

                                                                                                                                                                    xV

                                                                                                                                                                    V

                                                                                                                                                                    VxPV vxVv

                                                                                                                                                                    min

                                                                                                                                                                    xPV

                                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    d VB

                                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                                    tVVV

                                                                                                                                                                    1

                                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    d VB

                                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                                    tVVV

                                                                                                                                                                    1

                                                                                                                                                                    tVV

                                                                                                                                                                    tVVV BBBBP

                                                                                                                                                                    1

                                                                                                                                                                    Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                    bull General Solution in for Basis Matrix

                                                                                                                                                                    bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                    (thus projection is another linear operation)

                                                                                                                                                                    (note same operation underlies least squares)

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    d VB

                                                                                                                                                                    xBBBBxP tVV

                                                                                                                                                                    tVVV

                                                                                                                                                                    1

                                                                                                                                                                    tVV

                                                                                                                                                                    tVVV BBBBP

                                                                                                                                                                    1

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                                    nnVtV IBB

                                                                                                                                                                    10

                                                                                                                                                                    01

                                                                                                                                                                    1

                                                                                                                                                                    111

                                                                                                                                                                    1

                                                                                                                                                                    1

                                                                                                                                                                    nnn

                                                                                                                                                                    n

                                                                                                                                                                    ntn

                                                                                                                                                                    t

                                                                                                                                                                    vvvv

                                                                                                                                                                    vvvv

                                                                                                                                                                    vv

                                                                                                                                                                    v

                                                                                                                                                                    v

                                                                                                                                                                    nvv 1

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                                    bull So =

                                                                                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                    (Recall Right Multrsquon)

                                                                                                                                                                    nnVtV IBB

                                                                                                                                                                    10

                                                                                                                                                                    01

                                                                                                                                                                    1

                                                                                                                                                                    111

                                                                                                                                                                    1

                                                                                                                                                                    1

                                                                                                                                                                    nnn

                                                                                                                                                                    n

                                                                                                                                                                    ntn

                                                                                                                                                                    t

                                                                                                                                                                    vvvv

                                                                                                                                                                    vvvv

                                                                                                                                                                    vv

                                                                                                                                                                    v

                                                                                                                                                                    v

                                                                                                                                                                    xBBxP tVVV

                                                                                                                                                                    x V

                                                                                                                                                                    nvv 1

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                                    and

                                                                                                                                                                    V

                                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                                    and

                                                                                                                                                                    bull Parseval Inequality

                                                                                                                                                                    V

                                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                                    2

                                                                                                                                                                    1

                                                                                                                                                                    22

                                                                                                                                                                    1

                                                                                                                                                                    22 aavxxxP

                                                                                                                                                                    n

                                                                                                                                                                    ii

                                                                                                                                                                    n

                                                                                                                                                                    iiV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                    ddU IUU t

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                    ddU IUU t

                                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                                    i ii 2

                                                                                                                                                                    1

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                    ddU IUU t

                                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                                    i ii 2

                                                                                                                                                                    1

                                                                                                                                                                    U

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                    For a Matrix

                                                                                                                                                                    Find

                                                                                                                                                                    ndX

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                    For a Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    with Entries

                                                                                                                                                                    called Singular Values

                                                                                                                                                                    ndX

                                                                                                                                                                    ndS

                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                    For a Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    with Entries

                                                                                                                                                                    called Singular Values

                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                    (recall )

                                                                                                                                                                    ndX

                                                                                                                                                                    ndS

                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                    ddU nnV

                                                                                                                                                                    IVVUU tt

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                    For a Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    with Entries

                                                                                                                                                                    called Singular Values

                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                    (recall )

                                                                                                                                                                    So That

                                                                                                                                                                    ndX

                                                                                                                                                                    ndS

                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                    ddU nnV

                                                                                                                                                                    IVVUU tt tUSVX

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                    bull First rotate

                                                                                                                                                                    X

                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                    bull First rotate

                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                    X

                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                    is

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                    bull First rotate

                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                    X

                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                    is

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                    bull First rotate

                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                                                                    X

                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                    is

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    Useful Labeling

                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    Useful Labeling

                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    r

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    Useful Labeling

                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    r

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    Useful Labeling

                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                    Then

                                                                                                                                                                    Where are truncations of

                                                                                                                                                                    trnrrrd VSUX

                                                                                                                                                                    VSU

                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                    =

                                                                                                                                                                    Graphics Display Assumes

                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                    tV

                                                                                                                                                                    nd

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                    =

                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                    tV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                    =

                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                    All 0s in Bottom

                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                    tV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                    =

                                                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                                                    ndX ddU nnS nn

                                                                                                                                                                    tV

                                                                                                                                                                    nnd 0

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                                    tV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                    =

                                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                                    tV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    = ndX rdU

                                                                                                                                                                    rrS nrtV

                                                                                                                                                                    0

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    =

                                                                                                                                                                    These Get 0ed Out

                                                                                                                                                                    ndX rdU

                                                                                                                                                                    rrS nrtV

                                                                                                                                                                    0

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                    = ndX rdU

                                                                                                                                                                    rrS nrtV

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                    Find

                                                                                                                                                                    ddX

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    Called Eigenvalues

                                                                                                                                                                    Convenient Ordering

                                                                                                                                                                    ddX

                                                                                                                                                                    d

                                                                                                                                                                    D

                                                                                                                                                                    0

                                                                                                                                                                    01

                                                                                                                                                                    n 1

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                    (ie )

                                                                                                                                                                    ddX

                                                                                                                                                                    d

                                                                                                                                                                    D

                                                                                                                                                                    0

                                                                                                                                                                    01

                                                                                                                                                                    ddB

                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                    (ie )

                                                                                                                                                                    So that ie

                                                                                                                                                                    ddX

                                                                                                                                                                    d

                                                                                                                                                                    D

                                                                                                                                                                    0

                                                                                                                                                                    01

                                                                                                                                                                    ddB

                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                    (looks similar)

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                    Complex (uses )

                                                                                                                                                                    VU

                                                                                                                                                                    1i

                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                    VU

                                                                                                                                                                    X

                                                                                                                                                                    BVU

                                                                                                                                                                    1i

                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                    • SiZer Background
                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Parseval identity for

                                                                                                                                                                      in subsp genrsquod by o n basis

                                                                                                                                                                      bull Pythagorean theorem

                                                                                                                                                                      bull ldquoDecomposition of Energyrdquo

                                                                                                                                                                      bull ANOVA - sums of squares

                                                                                                                                                                      bull Transform has same length as

                                                                                                                                                                      ie ldquorotation in rdquo

                                                                                                                                                                      x

                                                                                                                                                                      nvv 1

                                                                                                                                                                      2

                                                                                                                                                                      1

                                                                                                                                                                      22

                                                                                                                                                                      1

                                                                                                                                                                      2 aavxx

                                                                                                                                                                      n

                                                                                                                                                                      ii

                                                                                                                                                                      n

                                                                                                                                                                      ii

                                                                                                                                                                      a xd

                                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                                      xV

                                                                                                                                                                      V

                                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                      bull Find that Solves

                                                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                                      xV

                                                                                                                                                                      V

                                                                                                                                                                      VxPV vxVv

                                                                                                                                                                      min

                                                                                                                                                                      Projection of a Vector onto a Subspace

                                                                                                                                                                      bull Idea Member of that is Closest to

                                                                                                                                                                      (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                      bull Find that Solves

                                                                                                                                                                      (ldquoLeast Squaresrdquo)

                                                                                                                                                                      bull For Inner Product (Hilbert) Space

                                                                                                                                                                      Exists and is Unique

                                                                                                                                                                      Review of Linear Algebra (Cont)x

                                                                                                                                                                      xV

                                                                                                                                                                      V

                                                                                                                                                                      VxPV vxVv

                                                                                                                                                                      min

                                                                                                                                                                      xPV

                                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      d VB

                                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                                      tVVV

                                                                                                                                                                      1

                                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      d VB

                                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                                      tVVV

                                                                                                                                                                      1

                                                                                                                                                                      tVV

                                                                                                                                                                      tVVV BBBBP

                                                                                                                                                                      1

                                                                                                                                                                      Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                      bull General Solution in for Basis Matrix

                                                                                                                                                                      bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                      (thus projection is another linear operation)

                                                                                                                                                                      (note same operation underlies least squares)

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      d VB

                                                                                                                                                                      xBBBBxP tVV

                                                                                                                                                                      tVVV

                                                                                                                                                                      1

                                                                                                                                                                      tVV

                                                                                                                                                                      tVVV BBBBP

                                                                                                                                                                      1

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                                                      nnVtV IBB

                                                                                                                                                                      10

                                                                                                                                                                      01

                                                                                                                                                                      1

                                                                                                                                                                      111

                                                                                                                                                                      1

                                                                                                                                                                      1

                                                                                                                                                                      nnn

                                                                                                                                                                      n

                                                                                                                                                                      ntn

                                                                                                                                                                      t

                                                                                                                                                                      vvvv

                                                                                                                                                                      vvvv

                                                                                                                                                                      vv

                                                                                                                                                                      v

                                                                                                                                                                      v

                                                                                                                                                                      nvv 1

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                                                      bull So =

                                                                                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                      (Recall Right Multrsquon)

                                                                                                                                                                      nnVtV IBB

                                                                                                                                                                      10

                                                                                                                                                                      01

                                                                                                                                                                      1

                                                                                                                                                                      111

                                                                                                                                                                      1

                                                                                                                                                                      1

                                                                                                                                                                      nnn

                                                                                                                                                                      n

                                                                                                                                                                      ntn

                                                                                                                                                                      t

                                                                                                                                                                      vvvv

                                                                                                                                                                      vvvv

                                                                                                                                                                      vv

                                                                                                                                                                      v

                                                                                                                                                                      v

                                                                                                                                                                      xBBxP tVVV

                                                                                                                                                                      x V

                                                                                                                                                                      nvv 1

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                                      and

                                                                                                                                                                      V

                                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                                      and

                                                                                                                                                                      bull Parseval Inequality

                                                                                                                                                                      V

                                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                                      2

                                                                                                                                                                      1

                                                                                                                                                                      22

                                                                                                                                                                      1

                                                                                                                                                                      22 aavxxxP

                                                                                                                                                                      n

                                                                                                                                                                      ii

                                                                                                                                                                      n

                                                                                                                                                                      iiV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                      ddU IUU t

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                      ddU IUU t

                                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                                      i ii 2

                                                                                                                                                                      1

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                      ddU IUU t

                                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                                      i ii 2

                                                                                                                                                                      1

                                                                                                                                                                      U

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                      For a Matrix

                                                                                                                                                                      Find

                                                                                                                                                                      ndX

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                      For a Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      with Entries

                                                                                                                                                                      called Singular Values

                                                                                                                                                                      ndX

                                                                                                                                                                      ndS

                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                      For a Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      with Entries

                                                                                                                                                                      called Singular Values

                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                      (recall )

                                                                                                                                                                      ndX

                                                                                                                                                                      ndS

                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                      ddU nnV

                                                                                                                                                                      IVVUU tt

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                      For a Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      with Entries

                                                                                                                                                                      called Singular Values

                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                      (recall )

                                                                                                                                                                      So That

                                                                                                                                                                      ndX

                                                                                                                                                                      ndS

                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                      ddU nnV

                                                                                                                                                                      IVVUU tt tUSVX

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                      bull First rotate

                                                                                                                                                                      X

                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                      bull First rotate

                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                      X

                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                      is

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                      bull First rotate

                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                      X

                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                      is

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                      bull First rotate

                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                                                                      X

                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                      is

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      Useful Labeling

                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      Useful Labeling

                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      r

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      Useful Labeling

                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      r

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      Useful Labeling

                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                      Then

                                                                                                                                                                      Where are truncations of

                                                                                                                                                                      trnrrrd VSUX

                                                                                                                                                                      VSU

                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                      =

                                                                                                                                                                      Graphics Display Assumes

                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                      tV

                                                                                                                                                                      nd

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                      =

                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                      tV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                      =

                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                      All 0s in Bottom

                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                      tV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                      =

                                                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                                                      ndX ddU nnS nn

                                                                                                                                                                      tV

                                                                                                                                                                      nnd 0

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                                                      tV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                      =

                                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                                      tV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      = ndX rdU

                                                                                                                                                                      rrS nrtV

                                                                                                                                                                      0

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      =

                                                                                                                                                                      These Get 0ed Out

                                                                                                                                                                      ndX rdU

                                                                                                                                                                      rrS nrtV

                                                                                                                                                                      0

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                      = ndX rdU

                                                                                                                                                                      rrS nrtV

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                      Find

                                                                                                                                                                      ddX

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      Called Eigenvalues

                                                                                                                                                                      Convenient Ordering

                                                                                                                                                                      ddX

                                                                                                                                                                      d

                                                                                                                                                                      D

                                                                                                                                                                      0

                                                                                                                                                                      01

                                                                                                                                                                      n 1

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                      (ie )

                                                                                                                                                                      ddX

                                                                                                                                                                      d

                                                                                                                                                                      D

                                                                                                                                                                      0

                                                                                                                                                                      01

                                                                                                                                                                      ddB

                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                      (ie )

                                                                                                                                                                      So that ie

                                                                                                                                                                      ddX

                                                                                                                                                                      d

                                                                                                                                                                      D

                                                                                                                                                                      0

                                                                                                                                                                      01

                                                                                                                                                                      ddB

                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                      DBBX tBDBX

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                      (looks similar)

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                      Complex (uses )

                                                                                                                                                                      VU

                                                                                                                                                                      1i

                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                      VU

                                                                                                                                                                      X

                                                                                                                                                                      BVU

                                                                                                                                                                      1i

                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                      • SiZer Background
                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                                        xV

                                                                                                                                                                        V

                                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                        bull Find that Solves

                                                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                                        xV

                                                                                                                                                                        V

                                                                                                                                                                        VxPV vxVv

                                                                                                                                                                        min

                                                                                                                                                                        Projection of a Vector onto a Subspace

                                                                                                                                                                        bull Idea Member of that is Closest to

                                                                                                                                                                        (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                        bull Find that Solves

                                                                                                                                                                        (ldquoLeast Squaresrdquo)

                                                                                                                                                                        bull For Inner Product (Hilbert) Space

                                                                                                                                                                        Exists and is Unique

                                                                                                                                                                        Review of Linear Algebra (Cont)x

                                                                                                                                                                        xV

                                                                                                                                                                        V

                                                                                                                                                                        VxPV vxVv

                                                                                                                                                                        min

                                                                                                                                                                        xPV

                                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        d VB

                                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                                        tVVV

                                                                                                                                                                        1

                                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        d VB

                                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                                        tVVV

                                                                                                                                                                        1

                                                                                                                                                                        tVV

                                                                                                                                                                        tVVV BBBBP

                                                                                                                                                                        1

                                                                                                                                                                        Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                        bull General Solution in for Basis Matrix

                                                                                                                                                                        bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                        (thus projection is another linear operation)

                                                                                                                                                                        (note same operation underlies least squares)

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        d VB

                                                                                                                                                                        xBBBBxP tVV

                                                                                                                                                                        tVVV

                                                                                                                                                                        1

                                                                                                                                                                        tVV

                                                                                                                                                                        tVVV BBBBP

                                                                                                                                                                        1

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                                                        nnVtV IBB

                                                                                                                                                                        10

                                                                                                                                                                        01

                                                                                                                                                                        1

                                                                                                                                                                        111

                                                                                                                                                                        1

                                                                                                                                                                        1

                                                                                                                                                                        nnn

                                                                                                                                                                        n

                                                                                                                                                                        ntn

                                                                                                                                                                        t

                                                                                                                                                                        vvvv

                                                                                                                                                                        vvvv

                                                                                                                                                                        vv

                                                                                                                                                                        v

                                                                                                                                                                        v

                                                                                                                                                                        nvv 1

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Projection using Orthonormal Basis

                                                                                                                                                                        bull Basis Matrix is Orthonormal

                                                                                                                                                                        bull So =

                                                                                                                                                                        = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                        (Recall Right Multrsquon)

                                                                                                                                                                        nnVtV IBB

                                                                                                                                                                        10

                                                                                                                                                                        01

                                                                                                                                                                        1

                                                                                                                                                                        111

                                                                                                                                                                        1

                                                                                                                                                                        1

                                                                                                                                                                        nnn

                                                                                                                                                                        n

                                                                                                                                                                        ntn

                                                                                                                                                                        t

                                                                                                                                                                        vvvv

                                                                                                                                                                        vvvv

                                                                                                                                                                        vv

                                                                                                                                                                        v

                                                                                                                                                                        v

                                                                                                                                                                        xBBxP tVVV

                                                                                                                                                                        x V

                                                                                                                                                                        nvv 1

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                                        and

                                                                                                                                                                        V

                                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                                        and

                                                                                                                                                                        bull Parseval Inequality

                                                                                                                                                                        V

                                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                                        2

                                                                                                                                                                        1

                                                                                                                                                                        22

                                                                                                                                                                        1

                                                                                                                                                                        22 aavxxxP

                                                                                                                                                                        n

                                                                                                                                                                        ii

                                                                                                                                                                        n

                                                                                                                                                                        iiV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                        ddU IUU t

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                        ddU IUU t

                                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                                        i ii 2

                                                                                                                                                                        1

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                        ddU IUU t

                                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                                        i ii 2

                                                                                                                                                                        1

                                                                                                                                                                        U

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                        For a Matrix

                                                                                                                                                                        Find

                                                                                                                                                                        ndX

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                        For a Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        with Entries

                                                                                                                                                                        called Singular Values

                                                                                                                                                                        ndX

                                                                                                                                                                        ndS

                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                        For a Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        with Entries

                                                                                                                                                                        called Singular Values

                                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                                        (recall )

                                                                                                                                                                        ndX

                                                                                                                                                                        ndS

                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                        ddU nnV

                                                                                                                                                                        IVVUU tt

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                        For a Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        with Entries

                                                                                                                                                                        called Singular Values

                                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                                        (recall )

                                                                                                                                                                        So That

                                                                                                                                                                        ndX

                                                                                                                                                                        ndS

                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                        ddU nnV

                                                                                                                                                                        IVVUU tt tUSVX

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                        bull First rotate

                                                                                                                                                                        X

                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                        bull First rotate

                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                        X

                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                        is

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                        bull First rotate

                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                        X

                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                        is

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                        bull First rotate

                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                                                                        X

                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                        is

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        Useful Labeling

                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        Useful Labeling

                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        r

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        Useful Labeling

                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        r

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        Useful Labeling

                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                        Then

                                                                                                                                                                        Where are truncations of

                                                                                                                                                                        trnrrrd VSUX

                                                                                                                                                                        VSU

                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                        =

                                                                                                                                                                        Graphics Display Assumes

                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                        tV

                                                                                                                                                                        nd

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                        =

                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                        tV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                        =

                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                        All 0s in Bottom

                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                        tV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                        =

                                                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                                                        ndX ddU nnS nn

                                                                                                                                                                        tV

                                                                                                                                                                        nnd 0

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                                                        tV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                        =

                                                                                                                                                                        Also Some of These May be 0

                                                                                                                                                                        ndX ndU nnS nn

                                                                                                                                                                        tV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        = ndX rdU

                                                                                                                                                                        rrS nrtV

                                                                                                                                                                        0

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        =

                                                                                                                                                                        These Get 0ed Out

                                                                                                                                                                        ndX rdU

                                                                                                                                                                        rrS nrtV

                                                                                                                                                                        0

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                        = ndX rdU

                                                                                                                                                                        rrS nrtV

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                        Find

                                                                                                                                                                        ddX

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        Called Eigenvalues

                                                                                                                                                                        Convenient Ordering

                                                                                                                                                                        ddX

                                                                                                                                                                        d

                                                                                                                                                                        D

                                                                                                                                                                        0

                                                                                                                                                                        01

                                                                                                                                                                        n 1

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                        (ie )

                                                                                                                                                                        ddX

                                                                                                                                                                        d

                                                                                                                                                                        D

                                                                                                                                                                        0

                                                                                                                                                                        01

                                                                                                                                                                        ddB

                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                        (ie )

                                                                                                                                                                        So that ie

                                                                                                                                                                        ddX

                                                                                                                                                                        d

                                                                                                                                                                        D

                                                                                                                                                                        0

                                                                                                                                                                        01

                                                                                                                                                                        ddB

                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                        DBBX tBDBX

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                        (looks similar)

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                        Complex (uses )

                                                                                                                                                                        VU

                                                                                                                                                                        1i

                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                        VU

                                                                                                                                                                        X

                                                                                                                                                                        BVU

                                                                                                                                                                        1i

                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                        • SiZer Background
                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                          bull Find that Solves

                                                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                                                          xV

                                                                                                                                                                          V

                                                                                                                                                                          VxPV vxVv

                                                                                                                                                                          min

                                                                                                                                                                          Projection of a Vector onto a Subspace

                                                                                                                                                                          bull Idea Member of that is Closest to

                                                                                                                                                                          (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                          bull Find that Solves

                                                                                                                                                                          (ldquoLeast Squaresrdquo)

                                                                                                                                                                          bull For Inner Product (Hilbert) Space

                                                                                                                                                                          Exists and is Unique

                                                                                                                                                                          Review of Linear Algebra (Cont)x

                                                                                                                                                                          xV

                                                                                                                                                                          V

                                                                                                                                                                          VxPV vxVv

                                                                                                                                                                          min

                                                                                                                                                                          xPV

                                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          d VB

                                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                                          tVVV

                                                                                                                                                                          1

                                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          d VB

                                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                                          tVVV

                                                                                                                                                                          1

                                                                                                                                                                          tVV

                                                                                                                                                                          tVVV BBBBP

                                                                                                                                                                          1

                                                                                                                                                                          Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                          bull General Solution in for Basis Matrix

                                                                                                                                                                          bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                          (thus projection is another linear operation)

                                                                                                                                                                          (note same operation underlies least squares)

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          d VB

                                                                                                                                                                          xBBBBxP tVV

                                                                                                                                                                          tVVV

                                                                                                                                                                          1

                                                                                                                                                                          tVV

                                                                                                                                                                          tVVV BBBBP

                                                                                                                                                                          1

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                                                          nnVtV IBB

                                                                                                                                                                          10

                                                                                                                                                                          01

                                                                                                                                                                          1

                                                                                                                                                                          111

                                                                                                                                                                          1

                                                                                                                                                                          1

                                                                                                                                                                          nnn

                                                                                                                                                                          n

                                                                                                                                                                          ntn

                                                                                                                                                                          t

                                                                                                                                                                          vvvv

                                                                                                                                                                          vvvv

                                                                                                                                                                          vv

                                                                                                                                                                          v

                                                                                                                                                                          v

                                                                                                                                                                          nvv 1

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Projection using Orthonormal Basis

                                                                                                                                                                          bull Basis Matrix is Orthonormal

                                                                                                                                                                          bull So =

                                                                                                                                                                          = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                          (Recall Right Multrsquon)

                                                                                                                                                                          nnVtV IBB

                                                                                                                                                                          10

                                                                                                                                                                          01

                                                                                                                                                                          1

                                                                                                                                                                          111

                                                                                                                                                                          1

                                                                                                                                                                          1

                                                                                                                                                                          nnn

                                                                                                                                                                          n

                                                                                                                                                                          ntn

                                                                                                                                                                          t

                                                                                                                                                                          vvvv

                                                                                                                                                                          vvvv

                                                                                                                                                                          vv

                                                                                                                                                                          v

                                                                                                                                                                          v

                                                                                                                                                                          xBBxP tVVV

                                                                                                                                                                          x V

                                                                                                                                                                          nvv 1

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                                                          and

                                                                                                                                                                          V

                                                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                                                          and

                                                                                                                                                                          bull Parseval Inequality

                                                                                                                                                                          V

                                                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                                                          2

                                                                                                                                                                          1

                                                                                                                                                                          22

                                                                                                                                                                          1

                                                                                                                                                                          22 aavxxxP

                                                                                                                                                                          n

                                                                                                                                                                          ii

                                                                                                                                                                          n

                                                                                                                                                                          iiV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                          ddU IUU t

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                          ddU IUU t

                                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                                          i ii 2

                                                                                                                                                                          1

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                          ddU IUU t

                                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                                          i ii 2

                                                                                                                                                                          1

                                                                                                                                                                          U

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                          For a Matrix

                                                                                                                                                                          Find

                                                                                                                                                                          ndX

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                          For a Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          with Entries

                                                                                                                                                                          called Singular Values

                                                                                                                                                                          ndX

                                                                                                                                                                          ndS

                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                          For a Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          with Entries

                                                                                                                                                                          called Singular Values

                                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                                          (recall )

                                                                                                                                                                          ndX

                                                                                                                                                                          ndS

                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                          ddU nnV

                                                                                                                                                                          IVVUU tt

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                          For a Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          with Entries

                                                                                                                                                                          called Singular Values

                                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                                          (recall )

                                                                                                                                                                          So That

                                                                                                                                                                          ndX

                                                                                                                                                                          ndS

                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                          ddU nnV

                                                                                                                                                                          IVVUU tt tUSVX

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                          bull First rotate

                                                                                                                                                                          X

                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                          bull First rotate

                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                          X

                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                          is

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                          bull First rotate

                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                          X

                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                          is

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                          bull First rotate

                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                                                                          X

                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                          is

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          Useful Labeling

                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          Useful Labeling

                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          r

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          Useful Labeling

                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          r

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          Useful Labeling

                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                          Then

                                                                                                                                                                          Where are truncations of

                                                                                                                                                                          trnrrrd VSUX

                                                                                                                                                                          VSU

                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                          =

                                                                                                                                                                          Graphics Display Assumes

                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                          tV

                                                                                                                                                                          nd

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                          =

                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                          tV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                          =

                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                          All 0s in Bottom

                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                          tV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                          =

                                                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                                                          ndX ddU nnS nn

                                                                                                                                                                          tV

                                                                                                                                                                          nnd 0

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                                                          tV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                          =

                                                                                                                                                                          Also Some of These May be 0

                                                                                                                                                                          ndX ndU nnS nn

                                                                                                                                                                          tV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          = ndX rdU

                                                                                                                                                                          rrS nrtV

                                                                                                                                                                          0

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          =

                                                                                                                                                                          These Get 0ed Out

                                                                                                                                                                          ndX rdU

                                                                                                                                                                          rrS nrtV

                                                                                                                                                                          0

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                          = ndX rdU

                                                                                                                                                                          rrS nrtV

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                          Find

                                                                                                                                                                          ddX

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          Called Eigenvalues

                                                                                                                                                                          Convenient Ordering

                                                                                                                                                                          ddX

                                                                                                                                                                          d

                                                                                                                                                                          D

                                                                                                                                                                          0

                                                                                                                                                                          01

                                                                                                                                                                          n 1

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                          (ie )

                                                                                                                                                                          ddX

                                                                                                                                                                          d

                                                                                                                                                                          D

                                                                                                                                                                          0

                                                                                                                                                                          01

                                                                                                                                                                          ddB

                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                          (ie )

                                                                                                                                                                          So that ie

                                                                                                                                                                          ddX

                                                                                                                                                                          d

                                                                                                                                                                          D

                                                                                                                                                                          0

                                                                                                                                                                          01

                                                                                                                                                                          ddB

                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                          DBBX tBDBX

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                          (looks similar)

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                          Complex (uses )

                                                                                                                                                                          VU

                                                                                                                                                                          1i

                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                          VU

                                                                                                                                                                          X

                                                                                                                                                                          BVU

                                                                                                                                                                          1i

                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                          • SiZer Background
                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                            Projection of a Vector onto a Subspace

                                                                                                                                                                            bull Idea Member of that is Closest to

                                                                                                                                                                            (ie ldquoBest Approxrsquonrdquo)

                                                                                                                                                                            bull Find that Solves

                                                                                                                                                                            (ldquoLeast Squaresrdquo)

                                                                                                                                                                            bull For Inner Product (Hilbert) Space

                                                                                                                                                                            Exists and is Unique

                                                                                                                                                                            Review of Linear Algebra (Cont)x

                                                                                                                                                                            xV

                                                                                                                                                                            V

                                                                                                                                                                            VxPV vxVv

                                                                                                                                                                            min

                                                                                                                                                                            xPV

                                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            d VB

                                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                                            tVVV

                                                                                                                                                                            1

                                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            d VB

                                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                                            tVVV

                                                                                                                                                                            1

                                                                                                                                                                            tVV

                                                                                                                                                                            tVVV BBBBP

                                                                                                                                                                            1

                                                                                                                                                                            Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                            bull General Solution in for Basis Matrix

                                                                                                                                                                            bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                            (thus projection is another linear operation)

                                                                                                                                                                            (note same operation underlies least squares)

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            d VB

                                                                                                                                                                            xBBBBxP tVV

                                                                                                                                                                            tVVV

                                                                                                                                                                            1

                                                                                                                                                                            tVV

                                                                                                                                                                            tVVV BBBBP

                                                                                                                                                                            1

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                                                            nnVtV IBB

                                                                                                                                                                            10

                                                                                                                                                                            01

                                                                                                                                                                            1

                                                                                                                                                                            111

                                                                                                                                                                            1

                                                                                                                                                                            1

                                                                                                                                                                            nnn

                                                                                                                                                                            n

                                                                                                                                                                            ntn

                                                                                                                                                                            t

                                                                                                                                                                            vvvv

                                                                                                                                                                            vvvv

                                                                                                                                                                            vv

                                                                                                                                                                            v

                                                                                                                                                                            v

                                                                                                                                                                            nvv 1

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Projection using Orthonormal Basis

                                                                                                                                                                            bull Basis Matrix is Orthonormal

                                                                                                                                                                            bull So =

                                                                                                                                                                            = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                            (Recall Right Multrsquon)

                                                                                                                                                                            nnVtV IBB

                                                                                                                                                                            10

                                                                                                                                                                            01

                                                                                                                                                                            1

                                                                                                                                                                            111

                                                                                                                                                                            1

                                                                                                                                                                            1

                                                                                                                                                                            nnn

                                                                                                                                                                            n

                                                                                                                                                                            ntn

                                                                                                                                                                            t

                                                                                                                                                                            vvvv

                                                                                                                                                                            vvvv

                                                                                                                                                                            vv

                                                                                                                                                                            v

                                                                                                                                                                            v

                                                                                                                                                                            xBBxP tVVV

                                                                                                                                                                            x V

                                                                                                                                                                            nvv 1

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                                                            and

                                                                                                                                                                            V

                                                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Projection using Orthonormal Basis (cont)

                                                                                                                                                                            bull For Orthogonal Complement

                                                                                                                                                                            and

                                                                                                                                                                            bull Parseval Inequality

                                                                                                                                                                            V

                                                                                                                                                                            xPxPx VV 222xPxPx VV

                                                                                                                                                                            2

                                                                                                                                                                            1

                                                                                                                                                                            22

                                                                                                                                                                            1

                                                                                                                                                                            22 aavxxxP

                                                                                                                                                                            n

                                                                                                                                                                            ii

                                                                                                                                                                            n

                                                                                                                                                                            iiV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                            ddU IUU t

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                            ddU IUU t

                                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                                            i ii 2

                                                                                                                                                                            1

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                            ddU IUU t

                                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                                            i ii 2

                                                                                                                                                                            1

                                                                                                                                                                            U

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                            For a Matrix

                                                                                                                                                                            Find

                                                                                                                                                                            ndX

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                            For a Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            with Entries

                                                                                                                                                                            called Singular Values

                                                                                                                                                                            ndX

                                                                                                                                                                            ndS

                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                            For a Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            with Entries

                                                                                                                                                                            called Singular Values

                                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                                            (recall )

                                                                                                                                                                            ndX

                                                                                                                                                                            ndS

                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                            ddU nnV

                                                                                                                                                                            IVVUU tt

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                            For a Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            with Entries

                                                                                                                                                                            called Singular Values

                                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                                            (recall )

                                                                                                                                                                            So That

                                                                                                                                                                            ndX

                                                                                                                                                                            ndS

                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                            ddU nnV

                                                                                                                                                                            IVVUU tt tUSVX

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                            bull First rotate

                                                                                                                                                                            X

                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                            bull First rotate

                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                            X

                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                            is

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                            bull First rotate

                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                            X

                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                            is

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                            bull First rotate

                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                                                                            X

                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                            is

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            Useful Labeling

                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            Useful Labeling

                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            r

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            Useful Labeling

                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            r

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            Useful Labeling

                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                            Then

                                                                                                                                                                            Where are truncations of

                                                                                                                                                                            trnrrrd VSUX

                                                                                                                                                                            VSU

                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                            =

                                                                                                                                                                            Graphics Display Assumes

                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                            tV

                                                                                                                                                                            nd

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                            =

                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                            tV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                            =

                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                            All 0s in Bottom

                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                            tV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                            =

                                                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                                                            ndX ddU nnS nn

                                                                                                                                                                            tV

                                                                                                                                                                            nnd 0

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                                                            tV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                            =

                                                                                                                                                                            Also Some of These May be 0

                                                                                                                                                                            ndX ndU nnS nn

                                                                                                                                                                            tV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            = ndX rdU

                                                                                                                                                                            rrS nrtV

                                                                                                                                                                            0

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            =

                                                                                                                                                                            These Get 0ed Out

                                                                                                                                                                            ndX rdU

                                                                                                                                                                            rrS nrtV

                                                                                                                                                                            0

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                            = ndX rdU

                                                                                                                                                                            rrS nrtV

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                            Find

                                                                                                                                                                            ddX

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            Called Eigenvalues

                                                                                                                                                                            Convenient Ordering

                                                                                                                                                                            ddX

                                                                                                                                                                            d

                                                                                                                                                                            D

                                                                                                                                                                            0

                                                                                                                                                                            01

                                                                                                                                                                            n 1

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                            (ie )

                                                                                                                                                                            ddX

                                                                                                                                                                            d

                                                                                                                                                                            D

                                                                                                                                                                            0

                                                                                                                                                                            01

                                                                                                                                                                            ddB

                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                            (ie )

                                                                                                                                                                            So that ie

                                                                                                                                                                            ddX

                                                                                                                                                                            d

                                                                                                                                                                            D

                                                                                                                                                                            0

                                                                                                                                                                            01

                                                                                                                                                                            ddB

                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                            DBBX tBDBX

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                            (looks similar)

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                            Complex (uses )

                                                                                                                                                                            VU

                                                                                                                                                                            1i

                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                            VU

                                                                                                                                                                            X

                                                                                                                                                                            BVU

                                                                                                                                                                            1i

                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                            • SiZer Background
                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              d VB

                                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                                              tVVV

                                                                                                                                                                              1

                                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              d VB

                                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                                              tVVV

                                                                                                                                                                              1

                                                                                                                                                                              tVV

                                                                                                                                                                              tVVV BBBBP

                                                                                                                                                                              1

                                                                                                                                                                              Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                              bull General Solution in for Basis Matrix

                                                                                                                                                                              bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                              (thus projection is another linear operation)

                                                                                                                                                                              (note same operation underlies least squares)

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              d VB

                                                                                                                                                                              xBBBBxP tVV

                                                                                                                                                                              tVVV

                                                                                                                                                                              1

                                                                                                                                                                              tVV

                                                                                                                                                                              tVVV BBBBP

                                                                                                                                                                              1

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                                                              nnVtV IBB

                                                                                                                                                                              10

                                                                                                                                                                              01

                                                                                                                                                                              1

                                                                                                                                                                              111

                                                                                                                                                                              1

                                                                                                                                                                              1

                                                                                                                                                                              nnn

                                                                                                                                                                              n

                                                                                                                                                                              ntn

                                                                                                                                                                              t

                                                                                                                                                                              vvvv

                                                                                                                                                                              vvvv

                                                                                                                                                                              vv

                                                                                                                                                                              v

                                                                                                                                                                              v

                                                                                                                                                                              nvv 1

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Projection using Orthonormal Basis

                                                                                                                                                                              bull Basis Matrix is Orthonormal

                                                                                                                                                                              bull So =

                                                                                                                                                                              = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                              (Recall Right Multrsquon)

                                                                                                                                                                              nnVtV IBB

                                                                                                                                                                              10

                                                                                                                                                                              01

                                                                                                                                                                              1

                                                                                                                                                                              111

                                                                                                                                                                              1

                                                                                                                                                                              1

                                                                                                                                                                              nnn

                                                                                                                                                                              n

                                                                                                                                                                              ntn

                                                                                                                                                                              t

                                                                                                                                                                              vvvv

                                                                                                                                                                              vvvv

                                                                                                                                                                              vv

                                                                                                                                                                              v

                                                                                                                                                                              v

                                                                                                                                                                              xBBxP tVVV

                                                                                                                                                                              x V

                                                                                                                                                                              nvv 1

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                                                              and

                                                                                                                                                                              V

                                                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Projection using Orthonormal Basis (cont)

                                                                                                                                                                              bull For Orthogonal Complement

                                                                                                                                                                              and

                                                                                                                                                                              bull Parseval Inequality

                                                                                                                                                                              V

                                                                                                                                                                              xPxPx VV 222xPxPx VV

                                                                                                                                                                              2

                                                                                                                                                                              1

                                                                                                                                                                              22

                                                                                                                                                                              1

                                                                                                                                                                              22 aavxxxP

                                                                                                                                                                              n

                                                                                                                                                                              ii

                                                                                                                                                                              n

                                                                                                                                                                              iiV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                                              ddU IUU t

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                              ddU IUU t

                                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                                              i ii 2

                                                                                                                                                                              1

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                              ddU IUU t

                                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                                              i ii 2

                                                                                                                                                                              1

                                                                                                                                                                              U

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                              For a Matrix

                                                                                                                                                                              Find

                                                                                                                                                                              ndX

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                              For a Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              with Entries

                                                                                                                                                                              called Singular Values

                                                                                                                                                                              ndX

                                                                                                                                                                              ndS

                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                              For a Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              with Entries

                                                                                                                                                                              called Singular Values

                                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                                              (recall )

                                                                                                                                                                              ndX

                                                                                                                                                                              ndS

                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                              ddU nnV

                                                                                                                                                                              IVVUU tt

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                              For a Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              with Entries

                                                                                                                                                                              called Singular Values

                                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                                              (recall )

                                                                                                                                                                              So That

                                                                                                                                                                              ndX

                                                                                                                                                                              ndS

                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                              ddU nnV

                                                                                                                                                                              IVVUU tt tUSVX

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                              bull First rotate

                                                                                                                                                                              X

                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                              bull First rotate

                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                              X

                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                              is

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                              bull First rotate

                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                              X

                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                              is

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                              bull First rotate

                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                                                                              X

                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                              is

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              Useful Labeling

                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              Useful Labeling

                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              r

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              Useful Labeling

                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              r

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              Useful Labeling

                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                              Then

                                                                                                                                                                              Where are truncations of

                                                                                                                                                                              trnrrrd VSUX

                                                                                                                                                                              VSU

                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                              =

                                                                                                                                                                              Graphics Display Assumes

                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                              tV

                                                                                                                                                                              nd

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                              =

                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                              tV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                              =

                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                              All 0s in Bottom

                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                              tV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                              =

                                                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                                                              ndX ddU nnS nn

                                                                                                                                                                              tV

                                                                                                                                                                              nnd 0

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                                                              tV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                              =

                                                                                                                                                                              Also Some of These May be 0

                                                                                                                                                                              ndX ndU nnS nn

                                                                                                                                                                              tV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              = ndX rdU

                                                                                                                                                                              rrS nrtV

                                                                                                                                                                              0

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              =

                                                                                                                                                                              These Get 0ed Out

                                                                                                                                                                              ndX rdU

                                                                                                                                                                              rrS nrtV

                                                                                                                                                                              0

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                              = ndX rdU

                                                                                                                                                                              rrS nrtV

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                              Find

                                                                                                                                                                              ddX

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              Called Eigenvalues

                                                                                                                                                                              Convenient Ordering

                                                                                                                                                                              ddX

                                                                                                                                                                              d

                                                                                                                                                                              D

                                                                                                                                                                              0

                                                                                                                                                                              01

                                                                                                                                                                              n 1

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                              (ie )

                                                                                                                                                                              ddX

                                                                                                                                                                              d

                                                                                                                                                                              D

                                                                                                                                                                              0

                                                                                                                                                                              01

                                                                                                                                                                              ddB

                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                              (ie )

                                                                                                                                                                              So that ie

                                                                                                                                                                              ddX

                                                                                                                                                                              d

                                                                                                                                                                              D

                                                                                                                                                                              0

                                                                                                                                                                              01

                                                                                                                                                                              ddB

                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                              DBBX tBDBX

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                              (looks similar)

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                              Complex (uses )

                                                                                                                                                                              VU

                                                                                                                                                                              1i

                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                              VU

                                                                                                                                                                              X

                                                                                                                                                                              BVU

                                                                                                                                                                              1i

                                                                                                                                                                              • Participant Presentations
                                                                                                                                                                              • PCA to find clusters
                                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                                              • SiZer Background
                                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                                              • SiZer Overview
                                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                                              • New Subpopulation View
                                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                d VB

                                                                                                                                                                                xBBBBxP tVV

                                                                                                                                                                                tVVV

                                                                                                                                                                                1

                                                                                                                                                                                tVV

                                                                                                                                                                                tVVV BBBBP

                                                                                                                                                                                1

                                                                                                                                                                                Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                                bull General Solution in for Basis Matrix

                                                                                                                                                                                bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                                (thus projection is another linear operation)

                                                                                                                                                                                (note same operation underlies least squares)

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                d VB

                                                                                                                                                                                xBBBBxP tVV

                                                                                                                                                                                tVVV

                                                                                                                                                                                1

                                                                                                                                                                                tVV

                                                                                                                                                                                tVVV BBBBP

                                                                                                                                                                                1

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                                                nnVtV IBB

                                                                                                                                                                                10

                                                                                                                                                                                01

                                                                                                                                                                                1

                                                                                                                                                                                111

                                                                                                                                                                                1

                                                                                                                                                                                1

                                                                                                                                                                                nnn

                                                                                                                                                                                n

                                                                                                                                                                                ntn

                                                                                                                                                                                t

                                                                                                                                                                                vvvv

                                                                                                                                                                                vvvv

                                                                                                                                                                                vv

                                                                                                                                                                                v

                                                                                                                                                                                v

                                                                                                                                                                                nvv 1

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Projection using Orthonormal Basis

                                                                                                                                                                                bull Basis Matrix is Orthonormal

                                                                                                                                                                                bull So =

                                                                                                                                                                                = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                                (Recall Right Multrsquon)

                                                                                                                                                                                nnVtV IBB

                                                                                                                                                                                10

                                                                                                                                                                                01

                                                                                                                                                                                1

                                                                                                                                                                                111

                                                                                                                                                                                1

                                                                                                                                                                                1

                                                                                                                                                                                nnn

                                                                                                                                                                                n

                                                                                                                                                                                ntn

                                                                                                                                                                                t

                                                                                                                                                                                vvvv

                                                                                                                                                                                vvvv

                                                                                                                                                                                vv

                                                                                                                                                                                v

                                                                                                                                                                                v

                                                                                                                                                                                xBBxP tVVV

                                                                                                                                                                                x V

                                                                                                                                                                                nvv 1

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                                                and

                                                                                                                                                                                V

                                                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Projection using Orthonormal Basis (cont)

                                                                                                                                                                                bull For Orthogonal Complement

                                                                                                                                                                                and

                                                                                                                                                                                bull Parseval Inequality

                                                                                                                                                                                V

                                                                                                                                                                                xPxPx VV 222xPxPx VV

                                                                                                                                                                                2

                                                                                                                                                                                1

                                                                                                                                                                                22

                                                                                                                                                                                1

                                                                                                                                                                                22 aavxxxP

                                                                                                                                                                                n

                                                                                                                                                                                ii

                                                                                                                                                                                n

                                                                                                                                                                                iiV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                                ddU IUU t

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                ddU IUU t

                                                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                                                i ii 2

                                                                                                                                                                                1

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                ddU IUU t

                                                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                                                i ii 2

                                                                                                                                                                                1

                                                                                                                                                                                U

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                For a Matrix

                                                                                                                                                                                Find

                                                                                                                                                                                ndX

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                For a Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                with Entries

                                                                                                                                                                                called Singular Values

                                                                                                                                                                                ndX

                                                                                                                                                                                ndS

                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                For a Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                with Entries

                                                                                                                                                                                called Singular Values

                                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                                (recall )

                                                                                                                                                                                ndX

                                                                                                                                                                                ndS

                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                ddU nnV

                                                                                                                                                                                IVVUU tt

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                For a Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                with Entries

                                                                                                                                                                                called Singular Values

                                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                                (recall )

                                                                                                                                                                                So That

                                                                                                                                                                                ndX

                                                                                                                                                                                ndS

                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                ddU nnV

                                                                                                                                                                                IVVUU tt tUSVX

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                bull First rotate

                                                                                                                                                                                X

                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                bull First rotate

                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                X

                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                is

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                bull First rotate

                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                X

                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                is

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                bull First rotate

                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                                                                X

                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                is

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                r

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                r

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                Then

                                                                                                                                                                                Where are truncations of

                                                                                                                                                                                trnrrrd VSUX

                                                                                                                                                                                VSU

                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                =

                                                                                                                                                                                Graphics Display Assumes

                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                tV

                                                                                                                                                                                nd

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                =

                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                tV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                =

                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                All 0s in Bottom

                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                tV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                =

                                                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                                                tV

                                                                                                                                                                                nnd 0

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                                                tV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                =

                                                                                                                                                                                Also Some of These May be 0

                                                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                                                tV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                0

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                =

                                                                                                                                                                                These Get 0ed Out

                                                                                                                                                                                ndX rdU

                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                0

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                Find

                                                                                                                                                                                ddX

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                                Convenient Ordering

                                                                                                                                                                                ddX

                                                                                                                                                                                d

                                                                                                                                                                                D

                                                                                                                                                                                0

                                                                                                                                                                                01

                                                                                                                                                                                n 1

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                (ie )

                                                                                                                                                                                ddX

                                                                                                                                                                                d

                                                                                                                                                                                D

                                                                                                                                                                                0

                                                                                                                                                                                01

                                                                                                                                                                                ddB

                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                (ie )

                                                                                                                                                                                So that ie

                                                                                                                                                                                ddX

                                                                                                                                                                                d

                                                                                                                                                                                D

                                                                                                                                                                                0

                                                                                                                                                                                01

                                                                                                                                                                                ddB

                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                (looks similar)

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                Complex (uses )

                                                                                                                                                                                VU

                                                                                                                                                                                1i

                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                VU

                                                                                                                                                                                X

                                                                                                                                                                                BVU

                                                                                                                                                                                1i

                                                                                                                                                                                • Participant Presentations
                                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                                • SiZer Background
                                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                                • SiZer Overview
                                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                  Projection of a Vector onto a Subspace (cont)

                                                                                                                                                                                  bull General Solution in for Basis Matrix

                                                                                                                                                                                  bull So Projrsquon Operator is Matrix Multrsquon

                                                                                                                                                                                  (thus projection is another linear operation)

                                                                                                                                                                                  (note same operation underlies least squares)

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  d VB

                                                                                                                                                                                  xBBBBxP tVV

                                                                                                                                                                                  tVVV

                                                                                                                                                                                  1

                                                                                                                                                                                  tVV

                                                                                                                                                                                  tVVV BBBBP

                                                                                                                                                                                  1

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                                                  nnVtV IBB

                                                                                                                                                                                  10

                                                                                                                                                                                  01

                                                                                                                                                                                  1

                                                                                                                                                                                  111

                                                                                                                                                                                  1

                                                                                                                                                                                  1

                                                                                                                                                                                  nnn

                                                                                                                                                                                  n

                                                                                                                                                                                  ntn

                                                                                                                                                                                  t

                                                                                                                                                                                  vvvv

                                                                                                                                                                                  vvvv

                                                                                                                                                                                  vv

                                                                                                                                                                                  v

                                                                                                                                                                                  v

                                                                                                                                                                                  nvv 1

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Projection using Orthonormal Basis

                                                                                                                                                                                  bull Basis Matrix is Orthonormal

                                                                                                                                                                                  bull So =

                                                                                                                                                                                  = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                                  (Recall Right Multrsquon)

                                                                                                                                                                                  nnVtV IBB

                                                                                                                                                                                  10

                                                                                                                                                                                  01

                                                                                                                                                                                  1

                                                                                                                                                                                  111

                                                                                                                                                                                  1

                                                                                                                                                                                  1

                                                                                                                                                                                  nnn

                                                                                                                                                                                  n

                                                                                                                                                                                  ntn

                                                                                                                                                                                  t

                                                                                                                                                                                  vvvv

                                                                                                                                                                                  vvvv

                                                                                                                                                                                  vv

                                                                                                                                                                                  v

                                                                                                                                                                                  v

                                                                                                                                                                                  xBBxP tVVV

                                                                                                                                                                                  x V

                                                                                                                                                                                  nvv 1

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                                                  and

                                                                                                                                                                                  V

                                                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Projection using Orthonormal Basis (cont)

                                                                                                                                                                                  bull For Orthogonal Complement

                                                                                                                                                                                  and

                                                                                                                                                                                  bull Parseval Inequality

                                                                                                                                                                                  V

                                                                                                                                                                                  xPxPx VV 222xPxPx VV

                                                                                                                                                                                  2

                                                                                                                                                                                  1

                                                                                                                                                                                  22

                                                                                                                                                                                  1

                                                                                                                                                                                  22 aavxxxP

                                                                                                                                                                                  n

                                                                                                                                                                                  ii

                                                                                                                                                                                  n

                                                                                                                                                                                  iiV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                                  ddU IUU t

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                  ddU IUU t

                                                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                                                  i ii 2

                                                                                                                                                                                  1

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  (Real) Unitary Matrices with

                                                                                                                                                                                  bull Orthonormal Basis Matrix

                                                                                                                                                                                  (So All of Above Applies)

                                                                                                                                                                                  bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                  bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                  bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                  ddU IUU t

                                                                                                                                                                                  yxdyxyxyxUyUxUdn

                                                                                                                                                                                  i ii 2

                                                                                                                                                                                  1

                                                                                                                                                                                  U

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                  Find

                                                                                                                                                                                  ndX

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  with Entries

                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                  ndX

                                                                                                                                                                                  ndS

                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  with Entries

                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                                  (recall )

                                                                                                                                                                                  ndX

                                                                                                                                                                                  ndS

                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                  ddU nnV

                                                                                                                                                                                  IVVUU tt

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  with Entries

                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                                  (recall )

                                                                                                                                                                                  So That

                                                                                                                                                                                  ndX

                                                                                                                                                                                  ndS

                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                  ddU nnV

                                                                                                                                                                                  IVVUU tt tUSVX

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                  X

                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                  X

                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                  is

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                                  X

                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                  is

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                                                                  X

                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                  is

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  r

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  r

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                  Then

                                                                                                                                                                                  Where are truncations of

                                                                                                                                                                                  trnrrrd VSUX

                                                                                                                                                                                  VSU

                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  Graphics Display Assumes

                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  nd

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                  All 0s in Bottom

                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  nnd 0

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                                                  tV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                  0

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  =

                                                                                                                                                                                  These Get 0ed Out

                                                                                                                                                                                  ndX rdU

                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                  0

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                  Find

                                                                                                                                                                                  ddX

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  Called Eigenvalues

                                                                                                                                                                                  Convenient Ordering

                                                                                                                                                                                  ddX

                                                                                                                                                                                  d

                                                                                                                                                                                  D

                                                                                                                                                                                  0

                                                                                                                                                                                  01

                                                                                                                                                                                  n 1

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                  (ie )

                                                                                                                                                                                  ddX

                                                                                                                                                                                  d

                                                                                                                                                                                  D

                                                                                                                                                                                  0

                                                                                                                                                                                  01

                                                                                                                                                                                  ddB

                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                  (ie )

                                                                                                                                                                                  So that ie

                                                                                                                                                                                  ddX

                                                                                                                                                                                  d

                                                                                                                                                                                  D

                                                                                                                                                                                  0

                                                                                                                                                                                  01

                                                                                                                                                                                  ddB

                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                  (looks similar)

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                  Complex (uses )

                                                                                                                                                                                  VU

                                                                                                                                                                                  1i

                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                  VU

                                                                                                                                                                                  X

                                                                                                                                                                                  BVU

                                                                                                                                                                                  1i

                                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                                  • SiZer Background
                                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                                                    nnVtV IBB

                                                                                                                                                                                    10

                                                                                                                                                                                    01

                                                                                                                                                                                    1

                                                                                                                                                                                    111

                                                                                                                                                                                    1

                                                                                                                                                                                    1

                                                                                                                                                                                    nnn

                                                                                                                                                                                    n

                                                                                                                                                                                    ntn

                                                                                                                                                                                    t

                                                                                                                                                                                    vvvv

                                                                                                                                                                                    vvvv

                                                                                                                                                                                    vv

                                                                                                                                                                                    v

                                                                                                                                                                                    v

                                                                                                                                                                                    nvv 1

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Projection using Orthonormal Basis

                                                                                                                                                                                    bull Basis Matrix is Orthonormal

                                                                                                                                                                                    bull So =

                                                                                                                                                                                    = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                                    (Recall Right Multrsquon)

                                                                                                                                                                                    nnVtV IBB

                                                                                                                                                                                    10

                                                                                                                                                                                    01

                                                                                                                                                                                    1

                                                                                                                                                                                    111

                                                                                                                                                                                    1

                                                                                                                                                                                    1

                                                                                                                                                                                    nnn

                                                                                                                                                                                    n

                                                                                                                                                                                    ntn

                                                                                                                                                                                    t

                                                                                                                                                                                    vvvv

                                                                                                                                                                                    vvvv

                                                                                                                                                                                    vv

                                                                                                                                                                                    v

                                                                                                                                                                                    v

                                                                                                                                                                                    xBBxP tVVV

                                                                                                                                                                                    x V

                                                                                                                                                                                    nvv 1

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                                                    and

                                                                                                                                                                                    V

                                                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Projection using Orthonormal Basis (cont)

                                                                                                                                                                                    bull For Orthogonal Complement

                                                                                                                                                                                    and

                                                                                                                                                                                    bull Parseval Inequality

                                                                                                                                                                                    V

                                                                                                                                                                                    xPxPx VV 222xPxPx VV

                                                                                                                                                                                    2

                                                                                                                                                                                    1

                                                                                                                                                                                    22

                                                                                                                                                                                    1

                                                                                                                                                                                    22 aavxxxP

                                                                                                                                                                                    n

                                                                                                                                                                                    ii

                                                                                                                                                                                    n

                                                                                                                                                                                    iiV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                                    ddU IUU t

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                    ddU IUU t

                                                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                                                    i ii 2

                                                                                                                                                                                    1

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    (Real) Unitary Matrices with

                                                                                                                                                                                    bull Orthonormal Basis Matrix

                                                                                                                                                                                    (So All of Above Applies)

                                                                                                                                                                                    bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                    bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                    bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                    ddU IUU t

                                                                                                                                                                                    yxdyxyxyxUyUxUdn

                                                                                                                                                                                    i ii 2

                                                                                                                                                                                    1

                                                                                                                                                                                    U

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                    Find

                                                                                                                                                                                    ndX

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    with Entries

                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                    ndX

                                                                                                                                                                                    ndS

                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    with Entries

                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                                    (recall )

                                                                                                                                                                                    ndX

                                                                                                                                                                                    ndS

                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                    ddU nnV

                                                                                                                                                                                    IVVUU tt

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    with Entries

                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                                    (recall )

                                                                                                                                                                                    So That

                                                                                                                                                                                    ndX

                                                                                                                                                                                    ndS

                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                    ddU nnV

                                                                                                                                                                                    IVVUU tt tUSVX

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                    X

                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                    X

                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                    is

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                                    X

                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                    is

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                                                                                    X

                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                    is

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    r

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    r

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                    Then

                                                                                                                                                                                    Where are truncations of

                                                                                                                                                                                    trnrrrd VSUX

                                                                                                                                                                                    VSU

                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    Graphics Display Assumes

                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    nd

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                    All 0s in Bottom

                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                                                                    ndX ddU nnS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    nnd 0

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                                                    tV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                    0

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    =

                                                                                                                                                                                    These Get 0ed Out

                                                                                                                                                                                    ndX rdU

                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                    0

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                    Find

                                                                                                                                                                                    ddX

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    Called Eigenvalues

                                                                                                                                                                                    Convenient Ordering

                                                                                                                                                                                    ddX

                                                                                                                                                                                    d

                                                                                                                                                                                    D

                                                                                                                                                                                    0

                                                                                                                                                                                    01

                                                                                                                                                                                    n 1

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                    (ie )

                                                                                                                                                                                    ddX

                                                                                                                                                                                    d

                                                                                                                                                                                    D

                                                                                                                                                                                    0

                                                                                                                                                                                    01

                                                                                                                                                                                    ddB

                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                    (ie )

                                                                                                                                                                                    So that ie

                                                                                                                                                                                    ddX

                                                                                                                                                                                    d

                                                                                                                                                                                    D

                                                                                                                                                                                    0

                                                                                                                                                                                    01

                                                                                                                                                                                    ddB

                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                    (looks similar)

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                    Complex (uses )

                                                                                                                                                                                    VU

                                                                                                                                                                                    1i

                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                    VU

                                                                                                                                                                                    X

                                                                                                                                                                                    BVU

                                                                                                                                                                                    1i

                                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                                    • SiZer Background
                                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Projection using Orthonormal Basis

                                                                                                                                                                                      bull Basis Matrix is Orthonormal

                                                                                                                                                                                      bull So =

                                                                                                                                                                                      = Recon(Coeffs of ldquoin Dirrsquonrdquo)

                                                                                                                                                                                      (Recall Right Multrsquon)

                                                                                                                                                                                      nnVtV IBB

                                                                                                                                                                                      10

                                                                                                                                                                                      01

                                                                                                                                                                                      1

                                                                                                                                                                                      111

                                                                                                                                                                                      1

                                                                                                                                                                                      1

                                                                                                                                                                                      nnn

                                                                                                                                                                                      n

                                                                                                                                                                                      ntn

                                                                                                                                                                                      t

                                                                                                                                                                                      vvvv

                                                                                                                                                                                      vvvv

                                                                                                                                                                                      vv

                                                                                                                                                                                      v

                                                                                                                                                                                      v

                                                                                                                                                                                      xBBxP tVVV

                                                                                                                                                                                      x V

                                                                                                                                                                                      nvv 1

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                                                      and

                                                                                                                                                                                      V

                                                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Projection using Orthonormal Basis (cont)

                                                                                                                                                                                      bull For Orthogonal Complement

                                                                                                                                                                                      and

                                                                                                                                                                                      bull Parseval Inequality

                                                                                                                                                                                      V

                                                                                                                                                                                      xPxPx VV 222xPxPx VV

                                                                                                                                                                                      2

                                                                                                                                                                                      1

                                                                                                                                                                                      22

                                                                                                                                                                                      1

                                                                                                                                                                                      22 aavxxxP

                                                                                                                                                                                      n

                                                                                                                                                                                      ii

                                                                                                                                                                                      n

                                                                                                                                                                                      iiV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                                      ddU IUU t

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                      ddU IUU t

                                                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                                                      i ii 2

                                                                                                                                                                                      1

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      (Real) Unitary Matrices with

                                                                                                                                                                                      bull Orthonormal Basis Matrix

                                                                                                                                                                                      (So All of Above Applies)

                                                                                                                                                                                      bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                      bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                      bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                      ddU IUU t

                                                                                                                                                                                      yxdyxyxyxUyUxUdn

                                                                                                                                                                                      i ii 2

                                                                                                                                                                                      1

                                                                                                                                                                                      U

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                      Find

                                                                                                                                                                                      ndX

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      with Entries

                                                                                                                                                                                      called Singular Values

                                                                                                                                                                                      ndX

                                                                                                                                                                                      ndS

                                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      with Entries

                                                                                                                                                                                      called Singular Values

                                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                                      (recall )

                                                                                                                                                                                      ndX

                                                                                                                                                                                      ndS

                                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                                      ddU nnV

                                                                                                                                                                                      IVVUU tt

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      with Entries

                                                                                                                                                                                      called Singular Values

                                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                                      (recall )

                                                                                                                                                                                      So That

                                                                                                                                                                                      ndX

                                                                                                                                                                                      ndS

                                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                                      ddU nnV

                                                                                                                                                                                      IVVUU tt tUSVX

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                      X

                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                      X

                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                      is

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                                      X

                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                      is

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                                                                                      X

                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                      is

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      r

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      r

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                      Then

                                                                                                                                                                                      Where are truncations of

                                                                                                                                                                                      trnrrrd VSUX

                                                                                                                                                                                      VSU

                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      Graphics Display Assumes

                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      nd

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                      All 0s in Bottom

                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                                                                      ndX ddU nnS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      nnd 0

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                                                      tV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                      0

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      =

                                                                                                                                                                                      These Get 0ed Out

                                                                                                                                                                                      ndX rdU

                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                      0

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                      Find

                                                                                                                                                                                      ddX

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      Called Eigenvalues

                                                                                                                                                                                      Convenient Ordering

                                                                                                                                                                                      ddX

                                                                                                                                                                                      d

                                                                                                                                                                                      D

                                                                                                                                                                                      0

                                                                                                                                                                                      01

                                                                                                                                                                                      n 1

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                      (ie )

                                                                                                                                                                                      ddX

                                                                                                                                                                                      d

                                                                                                                                                                                      D

                                                                                                                                                                                      0

                                                                                                                                                                                      01

                                                                                                                                                                                      ddB

                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                      (ie )

                                                                                                                                                                                      So that ie

                                                                                                                                                                                      ddX

                                                                                                                                                                                      d

                                                                                                                                                                                      D

                                                                                                                                                                                      0

                                                                                                                                                                                      01

                                                                                                                                                                                      ddB

                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                      DBBX tBDBX

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                      (looks similar)

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                      Complex (uses )

                                                                                                                                                                                      VU

                                                                                                                                                                                      1i

                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                      VU

                                                                                                                                                                                      X

                                                                                                                                                                                      BVU

                                                                                                                                                                                      1i

                                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                                      • SiZer Background
                                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                                                        and

                                                                                                                                                                                        V

                                                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Projection using Orthonormal Basis (cont)

                                                                                                                                                                                        bull For Orthogonal Complement

                                                                                                                                                                                        and

                                                                                                                                                                                        bull Parseval Inequality

                                                                                                                                                                                        V

                                                                                                                                                                                        xPxPx VV 222xPxPx VV

                                                                                                                                                                                        2

                                                                                                                                                                                        1

                                                                                                                                                                                        22

                                                                                                                                                                                        1

                                                                                                                                                                                        22 aavxxxP

                                                                                                                                                                                        n

                                                                                                                                                                                        ii

                                                                                                                                                                                        n

                                                                                                                                                                                        iiV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                                        ddU IUU t

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                        ddU IUU t

                                                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                                                        i ii 2

                                                                                                                                                                                        1

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        (Real) Unitary Matrices with

                                                                                                                                                                                        bull Orthonormal Basis Matrix

                                                                                                                                                                                        (So All of Above Applies)

                                                                                                                                                                                        bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                        bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                        bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                        ddU IUU t

                                                                                                                                                                                        yxdyxyxyxUyUxUdn

                                                                                                                                                                                        i ii 2

                                                                                                                                                                                        1

                                                                                                                                                                                        U

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                                        For a Matrix

                                                                                                                                                                                        Find

                                                                                                                                                                                        ndX

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                                        For a Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        with Entries

                                                                                                                                                                                        called Singular Values

                                                                                                                                                                                        ndX

                                                                                                                                                                                        ndS

                                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                                        For a Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        with Entries

                                                                                                                                                                                        called Singular Values

                                                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                                                        (recall )

                                                                                                                                                                                        ndX

                                                                                                                                                                                        ndS

                                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                                        ddU nnV

                                                                                                                                                                                        IVVUU tt

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                                        For a Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        with Entries

                                                                                                                                                                                        called Singular Values

                                                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                                                        (recall )

                                                                                                                                                                                        So That

                                                                                                                                                                                        ndX

                                                                                                                                                                                        ndS

                                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                                        ddU nnV

                                                                                                                                                                                        IVVUU tt tUSVX

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                        X

                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                        X

                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                        is

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                                        X

                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                        is

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                                                                                        X

                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                        is

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        r

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        r

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                        Then

                                                                                                                                                                                        Where are truncations of

                                                                                                                                                                                        trnrrrd VSUX

                                                                                                                                                                                        VSU

                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        Graphics Display Assumes

                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        nd

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                        All 0s in Bottom

                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                                                                        ndX ddU nnS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        nnd 0

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        Also Some of These May be 0

                                                                                                                                                                                        ndX ndU nnS nn

                                                                                                                                                                                        tV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                        0

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        =

                                                                                                                                                                                        These Get 0ed Out

                                                                                                                                                                                        ndX rdU

                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                        0

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                        Find

                                                                                                                                                                                        ddX

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        Called Eigenvalues

                                                                                                                                                                                        Convenient Ordering

                                                                                                                                                                                        ddX

                                                                                                                                                                                        d

                                                                                                                                                                                        D

                                                                                                                                                                                        0

                                                                                                                                                                                        01

                                                                                                                                                                                        n 1

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                        (ie )

                                                                                                                                                                                        ddX

                                                                                                                                                                                        d

                                                                                                                                                                                        D

                                                                                                                                                                                        0

                                                                                                                                                                                        01

                                                                                                                                                                                        ddB

                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                        (ie )

                                                                                                                                                                                        So that ie

                                                                                                                                                                                        ddX

                                                                                                                                                                                        d

                                                                                                                                                                                        D

                                                                                                                                                                                        0

                                                                                                                                                                                        01

                                                                                                                                                                                        ddB

                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                        DBBX tBDBX

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                        (looks similar)

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                        Complex (uses )

                                                                                                                                                                                        VU

                                                                                                                                                                                        1i

                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                        VU

                                                                                                                                                                                        X

                                                                                                                                                                                        BVU

                                                                                                                                                                                        1i

                                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                                        • SiZer Background
                                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Projection using Orthonormal Basis (cont)

                                                                                                                                                                                          bull For Orthogonal Complement

                                                                                                                                                                                          and

                                                                                                                                                                                          bull Parseval Inequality

                                                                                                                                                                                          V

                                                                                                                                                                                          xPxPx VV 222xPxPx VV

                                                                                                                                                                                          2

                                                                                                                                                                                          1

                                                                                                                                                                                          22

                                                                                                                                                                                          1

                                                                                                                                                                                          22 aavxxxP

                                                                                                                                                                                          n

                                                                                                                                                                                          ii

                                                                                                                                                                                          n

                                                                                                                                                                                          iiV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                                          ddU IUU t

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                          ddU IUU t

                                                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                                                          i ii 2

                                                                                                                                                                                          1

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          (Real) Unitary Matrices with

                                                                                                                                                                                          bull Orthonormal Basis Matrix

                                                                                                                                                                                          (So All of Above Applies)

                                                                                                                                                                                          bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                          bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                          bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                          ddU IUU t

                                                                                                                                                                                          yxdyxyxyxUyUxUdn

                                                                                                                                                                                          i ii 2

                                                                                                                                                                                          1

                                                                                                                                                                                          U

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                                          For a Matrix

                                                                                                                                                                                          Find

                                                                                                                                                                                          ndX

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                                          For a Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          with Entries

                                                                                                                                                                                          called Singular Values

                                                                                                                                                                                          ndX

                                                                                                                                                                                          ndS

                                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                                          For a Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          with Entries

                                                                                                                                                                                          called Singular Values

                                                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                                                          (recall )

                                                                                                                                                                                          ndX

                                                                                                                                                                                          ndS

                                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                                          ddU nnV

                                                                                                                                                                                          IVVUU tt

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Singular Value Decomposition (SVD)

                                                                                                                                                                                          For a Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          with Entries

                                                                                                                                                                                          called Singular Values

                                                                                                                                                                                          And Unitary (Rotation) Matrices

                                                                                                                                                                                          (recall )

                                                                                                                                                                                          So That

                                                                                                                                                                                          ndX

                                                                                                                                                                                          ndS

                                                                                                                                                                                          )min(1 ndss

                                                                                                                                                                                          ddU nnV

                                                                                                                                                                                          IVVUU tt tUSVX

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                          X

                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                          X

                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                          is

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                                          X

                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                          is

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                                                                                          X

                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                          is

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          r

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          r

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                                          Then

                                                                                                                                                                                          Where are truncations of

                                                                                                                                                                                          trnrrrd VSUX

                                                                                                                                                                                          VSU

                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          Graphics Display Assumes

                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          nd

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                          All 0s in Bottom

                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                                                                          ndX ddU nnS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          nnd 0

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          Also Some of These May be 0

                                                                                                                                                                                          ndX ndU nnS nn

                                                                                                                                                                                          tV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                          0

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          =

                                                                                                                                                                                          These Get 0ed Out

                                                                                                                                                                                          ndX rdU

                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                          0

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                          Find

                                                                                                                                                                                          ddX

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          Called Eigenvalues

                                                                                                                                                                                          Convenient Ordering

                                                                                                                                                                                          ddX

                                                                                                                                                                                          d

                                                                                                                                                                                          D

                                                                                                                                                                                          0

                                                                                                                                                                                          01

                                                                                                                                                                                          n 1

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                          (ie )

                                                                                                                                                                                          ddX

                                                                                                                                                                                          d

                                                                                                                                                                                          D

                                                                                                                                                                                          0

                                                                                                                                                                                          01

                                                                                                                                                                                          ddB

                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                          (ie )

                                                                                                                                                                                          So that ie

                                                                                                                                                                                          ddX

                                                                                                                                                                                          d

                                                                                                                                                                                          D

                                                                                                                                                                                          0

                                                                                                                                                                                          01

                                                                                                                                                                                          ddB

                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                          DBBX tBDBX

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                          (looks similar)

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                          Complex (uses )

                                                                                                                                                                                          VU

                                                                                                                                                                                          1i

                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                          VU

                                                                                                                                                                                          X

                                                                                                                                                                                          BVU

                                                                                                                                                                                          1i

                                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                                          • SiZer Background
                                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                                            ddU IUU t

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                            ddU IUU t

                                                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                                                            i ii 2

                                                                                                                                                                                            1

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            (Real) Unitary Matrices with

                                                                                                                                                                                            bull Orthonormal Basis Matrix

                                                                                                                                                                                            (So All of Above Applies)

                                                                                                                                                                                            bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                            bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                            bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                            ddU IUU t

                                                                                                                                                                                            yxdyxyxyxUyUxUdn

                                                                                                                                                                                            i ii 2

                                                                                                                                                                                            1

                                                                                                                                                                                            U

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                                            For a Matrix

                                                                                                                                                                                            Find

                                                                                                                                                                                            ndX

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                                            For a Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            with Entries

                                                                                                                                                                                            called Singular Values

                                                                                                                                                                                            ndX

                                                                                                                                                                                            ndS

                                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                                            For a Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            with Entries

                                                                                                                                                                                            called Singular Values

                                                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                                                            (recall )

                                                                                                                                                                                            ndX

                                                                                                                                                                                            ndS

                                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                                            ddU nnV

                                                                                                                                                                                            IVVUU tt

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Singular Value Decomposition (SVD)

                                                                                                                                                                                            For a Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            with Entries

                                                                                                                                                                                            called Singular Values

                                                                                                                                                                                            And Unitary (Rotation) Matrices

                                                                                                                                                                                            (recall )

                                                                                                                                                                                            So That

                                                                                                                                                                                            ndX

                                                                                                                                                                                            ndS

                                                                                                                                                                                            )min(1 ndss

                                                                                                                                                                                            ddU nnV

                                                                                                                                                                                            IVVUU tt tUSVX

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                            X

                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                            X

                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                            is

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                                            X

                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                            is

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                                                                                            X

                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                            is

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            r

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            r

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                                            Then

                                                                                                                                                                                            Where are truncations of

                                                                                                                                                                                            trnrrrd VSUX

                                                                                                                                                                                            VSU

                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            Graphics Display Assumes

                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            nd

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                            All 0s in Bottom

                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                                                                            ndX ddU nnS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            nnd 0

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            Also Some of These May be 0

                                                                                                                                                                                            ndX ndU nnS nn

                                                                                                                                                                                            tV

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                            0

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            =

                                                                                                                                                                                            These Get 0ed Out

                                                                                                                                                                                            ndX rdU

                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                            0

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                            Find

                                                                                                                                                                                            ddX

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            Called Eigenvalues

                                                                                                                                                                                            Convenient Ordering

                                                                                                                                                                                            ddX

                                                                                                                                                                                            d

                                                                                                                                                                                            D

                                                                                                                                                                                            0

                                                                                                                                                                                            01

                                                                                                                                                                                            n 1

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                            (ie )

                                                                                                                                                                                            ddX

                                                                                                                                                                                            d

                                                                                                                                                                                            D

                                                                                                                                                                                            0

                                                                                                                                                                                            01

                                                                                                                                                                                            ddB

                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                            (ie )

                                                                                                                                                                                            So that ie

                                                                                                                                                                                            ddX

                                                                                                                                                                                            d

                                                                                                                                                                                            D

                                                                                                                                                                                            0

                                                                                                                                                                                            01

                                                                                                                                                                                            ddB

                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                            DBBX tBDBX

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                            (looks similar)

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                            Complex (uses )

                                                                                                                                                                                            VU

                                                                                                                                                                                            1i

                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                            VU

                                                                                                                                                                                            X

                                                                                                                                                                                            BVU

                                                                                                                                                                                            1i

                                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                                            • SiZer Background
                                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                              ddU IUU t

                                                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                                                              i ii 2

                                                                                                                                                                                              1

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              (Real) Unitary Matrices with

                                                                                                                                                                                              bull Orthonormal Basis Matrix

                                                                                                                                                                                              (So All of Above Applies)

                                                                                                                                                                                              bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                              bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                              bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                              ddU IUU t

                                                                                                                                                                                              yxdyxyxyxUyUxUdn

                                                                                                                                                                                              i ii 2

                                                                                                                                                                                              1

                                                                                                                                                                                              U

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                                              For a Matrix

                                                                                                                                                                                              Find

                                                                                                                                                                                              ndX

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                                              For a Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              with Entries

                                                                                                                                                                                              called Singular Values

                                                                                                                                                                                              ndX

                                                                                                                                                                                              ndS

                                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                                              For a Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              with Entries

                                                                                                                                                                                              called Singular Values

                                                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                                                              (recall )

                                                                                                                                                                                              ndX

                                                                                                                                                                                              ndS

                                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                                              ddU nnV

                                                                                                                                                                                              IVVUU tt

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Singular Value Decomposition (SVD)

                                                                                                                                                                                              For a Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              with Entries

                                                                                                                                                                                              called Singular Values

                                                                                                                                                                                              And Unitary (Rotation) Matrices

                                                                                                                                                                                              (recall )

                                                                                                                                                                                              So That

                                                                                                                                                                                              ndX

                                                                                                                                                                                              ndS

                                                                                                                                                                                              )min(1 ndss

                                                                                                                                                                                              ddU nnV

                                                                                                                                                                                              IVVUU tt tUSVX

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                              X

                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                              X

                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                              is

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                                              X

                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                              is

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                                                                                              X

                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                              is

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              r

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              r

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                                              Then

                                                                                                                                                                                              Where are truncations of

                                                                                                                                                                                              trnrrrd VSUX

                                                                                                                                                                                              VSU

                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              Graphics Display Assumes

                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              nd

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                              All 0s in Bottom

                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                                                                              ndX ddU nnS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              nnd 0

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              Also Some of These May be 0

                                                                                                                                                                                              ndX ndU nnS nn

                                                                                                                                                                                              tV

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                              0

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              =

                                                                                                                                                                                              These Get 0ed Out

                                                                                                                                                                                              ndX rdU

                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                              0

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                              Find

                                                                                                                                                                                              ddX

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              Called Eigenvalues

                                                                                                                                                                                              Convenient Ordering

                                                                                                                                                                                              ddX

                                                                                                                                                                                              d

                                                                                                                                                                                              D

                                                                                                                                                                                              0

                                                                                                                                                                                              01

                                                                                                                                                                                              n 1

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                              (ie )

                                                                                                                                                                                              ddX

                                                                                                                                                                                              d

                                                                                                                                                                                              D

                                                                                                                                                                                              0

                                                                                                                                                                                              01

                                                                                                                                                                                              ddB

                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                              (ie )

                                                                                                                                                                                              So that ie

                                                                                                                                                                                              ddX

                                                                                                                                                                                              d

                                                                                                                                                                                              D

                                                                                                                                                                                              0

                                                                                                                                                                                              01

                                                                                                                                                                                              ddB

                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                              DBBX tBDBX

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                              (looks similar)

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                              Complex (uses )

                                                                                                                                                                                              VU

                                                                                                                                                                                              1i

                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                              VU

                                                                                                                                                                                              X

                                                                                                                                                                                              BVU

                                                                                                                                                                                              1i

                                                                                                                                                                                              • Participant Presentations
                                                                                                                                                                                              • PCA to find clusters
                                                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                                                              • SiZer Background
                                                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                                                              • SiZer Overview
                                                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                                                              • New Subpopulation View
                                                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                (Real) Unitary Matrices with

                                                                                                                                                                                                bull Orthonormal Basis Matrix

                                                                                                                                                                                                (So All of Above Applies)

                                                                                                                                                                                                bull Note Transformrsquon is Distance Preserving

                                                                                                                                                                                                bull Lin Trans (Mult by ) is ~ Rotation

                                                                                                                                                                                                bull But also Includes ldquoMirror Imagesrdquo

                                                                                                                                                                                                ddU IUU t

                                                                                                                                                                                                yxdyxyxyxUyUxUdn

                                                                                                                                                                                                i ii 2

                                                                                                                                                                                                1

                                                                                                                                                                                                U

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                                For a Matrix

                                                                                                                                                                                                Find

                                                                                                                                                                                                ndX

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                                For a Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                with Entries

                                                                                                                                                                                                called Singular Values

                                                                                                                                                                                                ndX

                                                                                                                                                                                                ndS

                                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                                For a Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                with Entries

                                                                                                                                                                                                called Singular Values

                                                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                                                (recall )

                                                                                                                                                                                                ndX

                                                                                                                                                                                                ndS

                                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                                ddU nnV

                                                                                                                                                                                                IVVUU tt

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Singular Value Decomposition (SVD)

                                                                                                                                                                                                For a Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                with Entries

                                                                                                                                                                                                called Singular Values

                                                                                                                                                                                                And Unitary (Rotation) Matrices

                                                                                                                                                                                                (recall )

                                                                                                                                                                                                So That

                                                                                                                                                                                                ndX

                                                                                                                                                                                                ndS

                                                                                                                                                                                                )min(1 ndss

                                                                                                                                                                                                ddU nnV

                                                                                                                                                                                                IVVUU tt tUSVX

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                X

                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                                X

                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                is

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                                X

                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                is

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                                                                                X

                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                is

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                r

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                r

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                                Then

                                                                                                                                                                                                Where are truncations of

                                                                                                                                                                                                trnrrrd VSUX

                                                                                                                                                                                                VSU

                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                Graphics Display Assumes

                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                nd

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                                All 0s in Bottom

                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                nnd 0

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                Also Some of These May be 0

                                                                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                                                                tV

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                0

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                =

                                                                                                                                                                                                These Get 0ed Out

                                                                                                                                                                                                ndX rdU

                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                0

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                Find

                                                                                                                                                                                                ddX

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                                                Convenient Ordering

                                                                                                                                                                                                ddX

                                                                                                                                                                                                d

                                                                                                                                                                                                D

                                                                                                                                                                                                0

                                                                                                                                                                                                01

                                                                                                                                                                                                n 1

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                (ie )

                                                                                                                                                                                                ddX

                                                                                                                                                                                                d

                                                                                                                                                                                                D

                                                                                                                                                                                                0

                                                                                                                                                                                                01

                                                                                                                                                                                                ddB

                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                (ie )

                                                                                                                                                                                                So that ie

                                                                                                                                                                                                ddX

                                                                                                                                                                                                d

                                                                                                                                                                                                D

                                                                                                                                                                                                0

                                                                                                                                                                                                01

                                                                                                                                                                                                ddB

                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                (looks similar)

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                Complex (uses )

                                                                                                                                                                                                VU

                                                                                                                                                                                                1i

                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                VU

                                                                                                                                                                                                X

                                                                                                                                                                                                BVU

                                                                                                                                                                                                1i

                                                                                                                                                                                                • Participant Presentations
                                                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                                                • SiZer Background
                                                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                                                • SiZer Overview
                                                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                                  Find

                                                                                                                                                                                                  ndX

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  with Entries

                                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                                  ndX

                                                                                                                                                                                                  ndS

                                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  with Entries

                                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                                                  (recall )

                                                                                                                                                                                                  ndX

                                                                                                                                                                                                  ndS

                                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                                  ddU nnV

                                                                                                                                                                                                  IVVUU tt

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Singular Value Decomposition (SVD)

                                                                                                                                                                                                  For a Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  with Entries

                                                                                                                                                                                                  called Singular Values

                                                                                                                                                                                                  And Unitary (Rotation) Matrices

                                                                                                                                                                                                  (recall )

                                                                                                                                                                                                  So That

                                                                                                                                                                                                  ndX

                                                                                                                                                                                                  ndS

                                                                                                                                                                                                  )min(1 ndss

                                                                                                                                                                                                  ddU nnV

                                                                                                                                                                                                  IVVUU tt tUSVX

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                                  X

                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                                  X

                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                  is

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                                                  X

                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                  is

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                                                                                  X

                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                  is

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  r

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  r

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                                  Then

                                                                                                                                                                                                  Where are truncations of

                                                                                                                                                                                                  trnrrrd VSUX

                                                                                                                                                                                                  VSU

                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  Graphics Display Assumes

                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  nd

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                                  All 0s in Bottom

                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  nnd 0

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                                                                  tV

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                  0

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  =

                                                                                                                                                                                                  These Get 0ed Out

                                                                                                                                                                                                  ndX rdU

                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                  0

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                  Find

                                                                                                                                                                                                  ddX

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  Called Eigenvalues

                                                                                                                                                                                                  Convenient Ordering

                                                                                                                                                                                                  ddX

                                                                                                                                                                                                  d

                                                                                                                                                                                                  D

                                                                                                                                                                                                  0

                                                                                                                                                                                                  01

                                                                                                                                                                                                  n 1

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                  ddX

                                                                                                                                                                                                  d

                                                                                                                                                                                                  D

                                                                                                                                                                                                  0

                                                                                                                                                                                                  01

                                                                                                                                                                                                  ddB

                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                  So that ie

                                                                                                                                                                                                  ddX

                                                                                                                                                                                                  d

                                                                                                                                                                                                  D

                                                                                                                                                                                                  0

                                                                                                                                                                                                  01

                                                                                                                                                                                                  ddB

                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                  (looks similar)

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                  Complex (uses )

                                                                                                                                                                                                  VU

                                                                                                                                                                                                  1i

                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                  VU

                                                                                                                                                                                                  X

                                                                                                                                                                                                  BVU

                                                                                                                                                                                                  1i

                                                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                                                  • SiZer Background
                                                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    with Entries

                                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                                    ndX

                                                                                                                                                                                                    ndS

                                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    with Entries

                                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                                                    (recall )

                                                                                                                                                                                                    ndX

                                                                                                                                                                                                    ndS

                                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                                    ddU nnV

                                                                                                                                                                                                    IVVUU tt

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Singular Value Decomposition (SVD)

                                                                                                                                                                                                    For a Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    with Entries

                                                                                                                                                                                                    called Singular Values

                                                                                                                                                                                                    And Unitary (Rotation) Matrices

                                                                                                                                                                                                    (recall )

                                                                                                                                                                                                    So That

                                                                                                                                                                                                    ndX

                                                                                                                                                                                                    ndS

                                                                                                                                                                                                    )min(1 ndss

                                                                                                                                                                                                    ddU nnV

                                                                                                                                                                                                    IVVUU tt tUSVX

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                                    X

                                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                                    X

                                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                                    is

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                                                    X

                                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                                    is

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Intuition behind Singular Value Decomposition

                                                                                                                                                                                                    bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                    bull First rotate

                                                                                                                                                                                                    bull Second rescale coordinate axes (by )

                                                                                                                                                                                                    bull Third rotate again

                                                                                                                                                                                                    bull ie have diagonalized the transformation

                                                                                                                                                                                                    X

                                                                                                                                                                                                    vVSUvVSUvX tt

                                                                                                                                                                                                    is

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    r

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    r

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                                    Then

                                                                                                                                                                                                    Where are truncations of

                                                                                                                                                                                                    trnrrrd VSUX

                                                                                                                                                                                                    VSU

                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    Graphics Display Assumes

                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    nd

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                                    All 0s in Bottom

                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                                                                                    ndX ddU nnS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    nnd 0

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                                                                    tV

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                    0

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    =

                                                                                                                                                                                                    These Get 0ed Out

                                                                                                                                                                                                    ndX rdU

                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                    0

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                    Find

                                                                                                                                                                                                    ddX

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    Called Eigenvalues

                                                                                                                                                                                                    Convenient Ordering

                                                                                                                                                                                                    ddX

                                                                                                                                                                                                    d

                                                                                                                                                                                                    D

                                                                                                                                                                                                    0

                                                                                                                                                                                                    01

                                                                                                                                                                                                    n 1

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                    ddX

                                                                                                                                                                                                    d

                                                                                                                                                                                                    D

                                                                                                                                                                                                    0

                                                                                                                                                                                                    01

                                                                                                                                                                                                    ddB

                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                    So that ie

                                                                                                                                                                                                    ddX

                                                                                                                                                                                                    d

                                                                                                                                                                                                    D

                                                                                                                                                                                                    0

                                                                                                                                                                                                    01

                                                                                                                                                                                                    ddB

                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                    (looks similar)

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                    Complex (uses )

                                                                                                                                                                                                    VU

                                                                                                                                                                                                    1i

                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                    VU

                                                                                                                                                                                                    X

                                                                                                                                                                                                    BVU

                                                                                                                                                                                                    1i

                                                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                                                    • SiZer Background
                                                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                      with Entries

                                                                                                                                                                                                      called Singular Values

                                                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                                                      (recall )

                                                                                                                                                                                                      ndX

                                                                                                                                                                                                      ndS

                                                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                                                      ddU nnV

                                                                                                                                                                                                      IVVUU tt

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Singular Value Decomposition (SVD)

                                                                                                                                                                                                      For a Matrix

                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                      with Entries

                                                                                                                                                                                                      called Singular Values

                                                                                                                                                                                                      And Unitary (Rotation) Matrices

                                                                                                                                                                                                      (recall )

                                                                                                                                                                                                      So That

                                                                                                                                                                                                      ndX

                                                                                                                                                                                                      ndS

                                                                                                                                                                                                      )min(1 ndss

                                                                                                                                                                                                      ddU nnV

                                                                                                                                                                                                      IVVUU tt tUSVX

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                                      X

                                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                                      X

                                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                                      is

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                                                      X

                                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                                      is

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Intuition behind Singular Value Decomposition

                                                                                                                                                                                                      bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                      bull First rotate

                                                                                                                                                                                                      bull Second rescale coordinate axes (by )

                                                                                                                                                                                                      bull Third rotate again

                                                                                                                                                                                                      bull ie have diagonalized the transformation

                                                                                                                                                                                                      X

                                                                                                                                                                                                      vVSUvVSUvX tt

                                                                                                                                                                                                      is

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      r

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      r

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                                      Then

                                                                                                                                                                                                      Where are truncations of

                                                                                                                                                                                                      trnrrrd VSUX

                                                                                                                                                                                                      VSU

                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      Graphics Display Assumes

                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      nd

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                                      All 0s in Bottom

                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                                                                                      ndX ddU nnS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      nnd 0

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                                                                      tV

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                      0

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      =

                                                                                                                                                                                                      These Get 0ed Out

                                                                                                                                                                                                      ndX rdU

                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                      0

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                      Find

                                                                                                                                                                                                      ddX

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                      Called Eigenvalues

                                                                                                                                                                                                      Convenient Ordering

                                                                                                                                                                                                      ddX

                                                                                                                                                                                                      d

                                                                                                                                                                                                      D

                                                                                                                                                                                                      0

                                                                                                                                                                                                      01

                                                                                                                                                                                                      n 1

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                      ddX

                                                                                                                                                                                                      d

                                                                                                                                                                                                      D

                                                                                                                                                                                                      0

                                                                                                                                                                                                      01

                                                                                                                                                                                                      ddB

                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                      So that ie

                                                                                                                                                                                                      ddX

                                                                                                                                                                                                      d

                                                                                                                                                                                                      D

                                                                                                                                                                                                      0

                                                                                                                                                                                                      01

                                                                                                                                                                                                      ddB

                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                      DBBX tBDBX

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                      (looks similar)

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                      Complex (uses )

                                                                                                                                                                                                      VU

                                                                                                                                                                                                      1i

                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                      VU

                                                                                                                                                                                                      X

                                                                                                                                                                                                      BVU

                                                                                                                                                                                                      1i

                                                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                                                      • SiZer Background
                                                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Singular Value Decomposition (SVD)

                                                                                                                                                                                                        For a Matrix

                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                        with Entries

                                                                                                                                                                                                        called Singular Values

                                                                                                                                                                                                        And Unitary (Rotation) Matrices

                                                                                                                                                                                                        (recall )

                                                                                                                                                                                                        So That

                                                                                                                                                                                                        ndX

                                                                                                                                                                                                        ndS

                                                                                                                                                                                                        )min(1 ndss

                                                                                                                                                                                                        ddU nnV

                                                                                                                                                                                                        IVVUU tt tUSVX

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                                        X

                                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                                        X

                                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                                        is

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                                                        X

                                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                                        is

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Intuition behind Singular Value Decomposition

                                                                                                                                                                                                        bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                        bull First rotate

                                                                                                                                                                                                        bull Second rescale coordinate axes (by )

                                                                                                                                                                                                        bull Third rotate again

                                                                                                                                                                                                        bull ie have diagonalized the transformation

                                                                                                                                                                                                        X

                                                                                                                                                                                                        vVSUvVSUvX tt

                                                                                                                                                                                                        is

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                                        (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        r

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        r

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                                        Then

                                                                                                                                                                                                        Where are truncations of

                                                                                                                                                                                                        trnrrrd VSUX

                                                                                                                                                                                                        VSU

                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        Graphics Display Assumes

                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        nd

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                                        All 0s in Bottom

                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                                                                                        ndX ddU nnS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        nnd 0

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        Also Some of These May be 0

                                                                                                                                                                                                        ndX ndU nnS nn

                                                                                                                                                                                                        tV

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                        0

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        =

                                                                                                                                                                                                        These Get 0ed Out

                                                                                                                                                                                                        ndX rdU

                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                        0

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                        Find

                                                                                                                                                                                                        ddX

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                        Called Eigenvalues

                                                                                                                                                                                                        Convenient Ordering

                                                                                                                                                                                                        ddX

                                                                                                                                                                                                        d

                                                                                                                                                                                                        D

                                                                                                                                                                                                        0

                                                                                                                                                                                                        01

                                                                                                                                                                                                        n 1

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                        ddX

                                                                                                                                                                                                        d

                                                                                                                                                                                                        D

                                                                                                                                                                                                        0

                                                                                                                                                                                                        01

                                                                                                                                                                                                        ddB

                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                        So that ie

                                                                                                                                                                                                        ddX

                                                                                                                                                                                                        d

                                                                                                                                                                                                        D

                                                                                                                                                                                                        0

                                                                                                                                                                                                        01

                                                                                                                                                                                                        ddB

                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                        DBBX tBDBX

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                        (looks similar)

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                        Complex (uses )

                                                                                                                                                                                                        VU

                                                                                                                                                                                                        1i

                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                        VU

                                                                                                                                                                                                        X

                                                                                                                                                                                                        BVU

                                                                                                                                                                                                        1i

                                                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                                                        • SiZer Background
                                                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)X

                                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                                          X

                                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                                          X

                                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                                          is

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                                                          X

                                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                                          is

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Intuition behind Singular Value Decomposition

                                                                                                                                                                                                          bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                          bull First rotate

                                                                                                                                                                                                          bull Second rescale coordinate axes (by )

                                                                                                                                                                                                          bull Third rotate again

                                                                                                                                                                                                          bull ie have diagonalized the transformation

                                                                                                                                                                                                          X

                                                                                                                                                                                                          vVSUvVSUvX tt

                                                                                                                                                                                                          is

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                                          (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          r

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          r

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                                                          Then

                                                                                                                                                                                                          Where are truncations of

                                                                                                                                                                                                          trnrrrd VSUX

                                                                                                                                                                                                          VSU

                                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          Graphics Display Assumes

                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          nd

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                                          All 0s in Bottom

                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                                                                                          ndX ddU nnS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          nnd 0

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          Also Some of These May be 0

                                                                                                                                                                                                          ndX ndU nnS nn

                                                                                                                                                                                                          tV

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                          0

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          =

                                                                                                                                                                                                          These Get 0ed Out

                                                                                                                                                                                                          ndX rdU

                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                          0

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                          Find

                                                                                                                                                                                                          ddX

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                          Called Eigenvalues

                                                                                                                                                                                                          Convenient Ordering

                                                                                                                                                                                                          ddX

                                                                                                                                                                                                          d

                                                                                                                                                                                                          D

                                                                                                                                                                                                          0

                                                                                                                                                                                                          01

                                                                                                                                                                                                          n 1

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                          ddX

                                                                                                                                                                                                          d

                                                                                                                                                                                                          D

                                                                                                                                                                                                          0

                                                                                                                                                                                                          01

                                                                                                                                                                                                          ddB

                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                          So that ie

                                                                                                                                                                                                          ddX

                                                                                                                                                                                                          d

                                                                                                                                                                                                          D

                                                                                                                                                                                                          0

                                                                                                                                                                                                          01

                                                                                                                                                                                                          ddB

                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                          DBBX tBDBX

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                          (looks similar)

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                          Complex (uses )

                                                                                                                                                                                                          VU

                                                                                                                                                                                                          1i

                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                          VU

                                                                                                                                                                                                          X

                                                                                                                                                                                                          BVU

                                                                                                                                                                                                          1i

                                                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                                                          • SiZer Background
                                                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                                            X

                                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                                            X

                                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                                            is

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                                                            X

                                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                                            is

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Intuition behind Singular Value Decomposition

                                                                                                                                                                                                            bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                            bull First rotate

                                                                                                                                                                                                            bull Second rescale coordinate axes (by )

                                                                                                                                                                                                            bull Third rotate again

                                                                                                                                                                                                            bull ie have diagonalized the transformation

                                                                                                                                                                                                            X

                                                                                                                                                                                                            vVSUvVSUvX tt

                                                                                                                                                                                                            is

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                                            (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            r

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            r

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            Useful Labeling

                                                                                                                                                                                                            Singular Values in Increasing Order

                                                                                                                                                                                                            Note singular values = 0 can be omitted

                                                                                                                                                                                                            Let = of positive singular values

                                                                                                                                                                                                            Then

                                                                                                                                                                                                            Where are truncations of

                                                                                                                                                                                                            trnrrrd VSUX

                                                                                                                                                                                                            VSU

                                                                                                                                                                                                            )min(1 dnss

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            Graphics Display Assumes

                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            nd

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                                            All 0s in Bottom

                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                                                                                            ndX ddU nnS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            nnd 0

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            Also Some of These May be 0

                                                                                                                                                                                                            ndX ndU nnS nn

                                                                                                                                                                                                            tV

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                            0

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            =

                                                                                                                                                                                                            These Get 0ed Out

                                                                                                                                                                                                            ndX rdU

                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                            0

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                            Find

                                                                                                                                                                                                            ddX

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                            Called Eigenvalues

                                                                                                                                                                                                            Convenient Ordering

                                                                                                                                                                                                            ddX

                                                                                                                                                                                                            d

                                                                                                                                                                                                            D

                                                                                                                                                                                                            0

                                                                                                                                                                                                            01

                                                                                                                                                                                                            n 1

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                            ddX

                                                                                                                                                                                                            d

                                                                                                                                                                                                            D

                                                                                                                                                                                                            0

                                                                                                                                                                                                            01

                                                                                                                                                                                                            ddB

                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                            So that ie

                                                                                                                                                                                                            ddX

                                                                                                                                                                                                            d

                                                                                                                                                                                                            D

                                                                                                                                                                                                            0

                                                                                                                                                                                                            01

                                                                                                                                                                                                            ddB

                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                            DBBX tBDBX

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                            (looks similar)

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                            Complex (uses )

                                                                                                                                                                                                            VU

                                                                                                                                                                                                            1i

                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                            VU

                                                                                                                                                                                                            X

                                                                                                                                                                                                            BVU

                                                                                                                                                                                                            1i

                                                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                                                            • SiZer Background
                                                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                                              X

                                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                                              is

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                                                              X

                                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                                              is

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Intuition behind Singular Value Decomposition

                                                                                                                                                                                                              bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                              bull First rotate

                                                                                                                                                                                                              bull Second rescale coordinate axes (by )

                                                                                                                                                                                                              bull Third rotate again

                                                                                                                                                                                                              bull ie have diagonalized the transformation

                                                                                                                                                                                                              X

                                                                                                                                                                                                              vVSUvVSUvX tt

                                                                                                                                                                                                              is

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                                              (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              r

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              r

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              Useful Labeling

                                                                                                                                                                                                              Singular Values in Increasing Order

                                                                                                                                                                                                              Note singular values = 0 can be omitted

                                                                                                                                                                                                              Let = of positive singular values

                                                                                                                                                                                                              Then

                                                                                                                                                                                                              Where are truncations of

                                                                                                                                                                                                              trnrrrd VSUX

                                                                                                                                                                                                              VSU

                                                                                                                                                                                                              )min(1 dnss

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              Graphics Display Assumes

                                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              nd

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                                              All 0s in Bottom

                                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                                                                                              ndX ddU nnS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              nnd 0

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              Also Some of These May be 0

                                                                                                                                                                                                              ndX ndU nnS nn

                                                                                                                                                                                                              tV

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                              0

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              =

                                                                                                                                                                                                              These Get 0ed Out

                                                                                                                                                                                                              ndX rdU

                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                              0

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                              Find

                                                                                                                                                                                                              ddX

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                              Called Eigenvalues

                                                                                                                                                                                                              Convenient Ordering

                                                                                                                                                                                                              ddX

                                                                                                                                                                                                              d

                                                                                                                                                                                                              D

                                                                                                                                                                                                              0

                                                                                                                                                                                                              01

                                                                                                                                                                                                              n 1

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                              ddX

                                                                                                                                                                                                              d

                                                                                                                                                                                                              D

                                                                                                                                                                                                              0

                                                                                                                                                                                                              01

                                                                                                                                                                                                              ddB

                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                              So that ie

                                                                                                                                                                                                              ddX

                                                                                                                                                                                                              d

                                                                                                                                                                                                              D

                                                                                                                                                                                                              0

                                                                                                                                                                                                              01

                                                                                                                                                                                                              ddB

                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                              DBBX tBDBX

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                              (looks similar)

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                              Complex (uses )

                                                                                                                                                                                                              VU

                                                                                                                                                                                                              1i

                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                              VU

                                                                                                                                                                                                              X

                                                                                                                                                                                                              BVU

                                                                                                                                                                                                              1i

                                                                                                                                                                                                              • Participant Presentations
                                                                                                                                                                                                              • PCA to find clusters
                                                                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                                                                              • SiZer Background
                                                                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                                                                              • SiZer Overview
                                                                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                                                                              • New Subpopulation View
                                                                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                                                X

                                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                                is

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Intuition behind Singular Value Decomposition

                                                                                                                                                                                                                bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                                bull First rotate

                                                                                                                                                                                                                bull Second rescale coordinate axes (by )

                                                                                                                                                                                                                bull Third rotate again

                                                                                                                                                                                                                bull ie have diagonalized the transformation

                                                                                                                                                                                                                X

                                                                                                                                                                                                                vVSUvVSUvX tt

                                                                                                                                                                                                                is

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                                (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                r

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                r

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                Useful Labeling

                                                                                                                                                                                                                Singular Values in Increasing Order

                                                                                                                                                                                                                Note singular values = 0 can be omitted

                                                                                                                                                                                                                Let = of positive singular values

                                                                                                                                                                                                                Then

                                                                                                                                                                                                                Where are truncations of

                                                                                                                                                                                                                trnrrrd VSUX

                                                                                                                                                                                                                VSU

                                                                                                                                                                                                                )min(1 dnss

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                Graphics Display Assumes

                                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                nd

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                                                All 0s in Bottom

                                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                nnd 0

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                Also Some of These May be 0

                                                                                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                                                                                tV

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                0

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                =

                                                                                                                                                                                                                These Get 0ed Out

                                                                                                                                                                                                                ndX rdU

                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                0

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                Find

                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                                                                Convenient Ordering

                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                d

                                                                                                                                                                                                                D

                                                                                                                                                                                                                0

                                                                                                                                                                                                                01

                                                                                                                                                                                                                n 1

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                d

                                                                                                                                                                                                                D

                                                                                                                                                                                                                0

                                                                                                                                                                                                                01

                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                So that ie

                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                d

                                                                                                                                                                                                                D

                                                                                                                                                                                                                0

                                                                                                                                                                                                                01

                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                (looks similar)

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                Complex (uses )

                                                                                                                                                                                                                VU

                                                                                                                                                                                                                1i

                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                VU

                                                                                                                                                                                                                X

                                                                                                                                                                                                                BVU

                                                                                                                                                                                                                1i

                                                                                                                                                                                                                • Participant Presentations
                                                                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                                                                • SiZer Background
                                                                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                                                                • SiZer Overview
                                                                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Intuition behind Singular Value Decomposition

                                                                                                                                                                                                                  bull For a ldquolinear transfrsquonrdquo (via matrix multirsquon)

                                                                                                                                                                                                                  bull First rotate

                                                                                                                                                                                                                  bull Second rescale coordinate axes (by )

                                                                                                                                                                                                                  bull Third rotate again

                                                                                                                                                                                                                  bull ie have diagonalized the transformation

                                                                                                                                                                                                                  X

                                                                                                                                                                                                                  vVSUvVSUvX tt

                                                                                                                                                                                                                  is

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                                  (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  r

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  r

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  Useful Labeling

                                                                                                                                                                                                                  Singular Values in Increasing Order

                                                                                                                                                                                                                  Note singular values = 0 can be omitted

                                                                                                                                                                                                                  Let = of positive singular values

                                                                                                                                                                                                                  Then

                                                                                                                                                                                                                  Where are truncations of

                                                                                                                                                                                                                  trnrrrd VSUX

                                                                                                                                                                                                                  VSU

                                                                                                                                                                                                                  )min(1 dnss

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  Graphics Display Assumes

                                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  nd

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Full Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  Full Rank Basis Matrix

                                                                                                                                                                                                                  All 0s in Bottom

                                                                                                                                                                                                                  ndX ddU ndS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  nnd 0

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                  0

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  =

                                                                                                                                                                                                                  These Get 0ed Out

                                                                                                                                                                                                                  ndX rdU

                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                  0

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                  Find

                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                  Called Eigenvalues

                                                                                                                                                                                                                  Convenient Ordering

                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                  d

                                                                                                                                                                                                                  D

                                                                                                                                                                                                                  0

                                                                                                                                                                                                                  01

                                                                                                                                                                                                                  n 1

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                  d

                                                                                                                                                                                                                  D

                                                                                                                                                                                                                  0

                                                                                                                                                                                                                  01

                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                  So that ie

                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                  d

                                                                                                                                                                                                                  D

                                                                                                                                                                                                                  0

                                                                                                                                                                                                                  01

                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                  (looks similar)

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                  Complex (uses )

                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                  X

                                                                                                                                                                                                                  BVU

                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                                                                  • SiZer Background
                                                                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                                    (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    r

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    r

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    Useful Labeling

                                                                                                                                                                                                                    Singular Values in Increasing Order

                                                                                                                                                                                                                    Note singular values = 0 can be omitted

                                                                                                                                                                                                                    Let = of positive singular values

                                                                                                                                                                                                                    Then

                                                                                                                                                                                                                    Where are truncations of

                                                                                                                                                                                                                    trnrrrd VSUX

                                                                                                                                                                                                                    VSU

                                                                                                                                                                                                                    )min(1 dnss

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    Graphics Display Assumes

                                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    nd

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Full Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    Full Rank Basis Matrix

                                                                                                                                                                                                                    All 0s in Bottom

                                                                                                                                                                                                                    ndX ddU ndS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    These Columns Get 0ed Out

                                                                                                                                                                                                                    ndX ddU nnS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    nnd 0

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                    0

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    =

                                                                                                                                                                                                                    These Get 0ed Out

                                                                                                                                                                                                                    ndX rdU

                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                    0

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                    Find

                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                    Called Eigenvalues

                                                                                                                                                                                                                    Convenient Ordering

                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                    d

                                                                                                                                                                                                                    D

                                                                                                                                                                                                                    0

                                                                                                                                                                                                                    01

                                                                                                                                                                                                                    n 1

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                    d

                                                                                                                                                                                                                    D

                                                                                                                                                                                                                    0

                                                                                                                                                                                                                    01

                                                                                                                                                                                                                    ddB

                                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                                    So that ie

                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                    d

                                                                                                                                                                                                                    D

                                                                                                                                                                                                                    0

                                                                                                                                                                                                                    01

                                                                                                                                                                                                                    ddB

                                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                    (looks similar)

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                    Complex (uses )

                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                    X

                                                                                                                                                                                                                    BVU

                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                                                                    • SiZer Background
                                                                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                                      (Since do ldquo0-Stretchingrdquo)

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      r

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      r

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      Useful Labeling

                                                                                                                                                                                                                      Singular Values in Increasing Order

                                                                                                                                                                                                                      Note singular values = 0 can be omitted

                                                                                                                                                                                                                      Let = of positive singular values

                                                                                                                                                                                                                      Then

                                                                                                                                                                                                                      Where are truncations of

                                                                                                                                                                                                                      trnrrrd VSUX

                                                                                                                                                                                                                      VSU

                                                                                                                                                                                                                      )min(1 dnss

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      Graphics Display Assumes

                                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      nd

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Full Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      Full Rank Basis Matrix

                                                                                                                                                                                                                      All 0s in Bottom

                                                                                                                                                                                                                      ndX ddU ndS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      These Columns Get 0ed Out

                                                                                                                                                                                                                      ndX ddU nnS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      nnd 0

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                                      =ndX ndU nnS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                      0

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      =

                                                                                                                                                                                                                      These Get 0ed Out

                                                                                                                                                                                                                      ndX rdU

                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                      0

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                      Find

                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                      Called Eigenvalues

                                                                                                                                                                                                                      Convenient Ordering

                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                      d

                                                                                                                                                                                                                      D

                                                                                                                                                                                                                      0

                                                                                                                                                                                                                      01

                                                                                                                                                                                                                      n 1

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                      d

                                                                                                                                                                                                                      D

                                                                                                                                                                                                                      0

                                                                                                                                                                                                                      01

                                                                                                                                                                                                                      ddB

                                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                                      So that ie

                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                      d

                                                                                                                                                                                                                      D

                                                                                                                                                                                                                      0

                                                                                                                                                                                                                      01

                                                                                                                                                                                                                      ddB

                                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                                      DBBX tBDBX

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                      (looks similar)

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                      Complex (uses )

                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                      X

                                                                                                                                                                                                                      BVU

                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                                                                      • SiZer Background
                                                                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        r

                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        r

                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                        Useful Labeling

                                                                                                                                                                                                                        Singular Values in Increasing Order

                                                                                                                                                                                                                        Note singular values = 0 can be omitted

                                                                                                                                                                                                                        Let = of positive singular values

                                                                                                                                                                                                                        Then

                                                                                                                                                                                                                        Where are truncations of

                                                                                                                                                                                                                        trnrrrd VSUX

                                                                                                                                                                                                                        VSU

                                                                                                                                                                                                                        )min(1 dnss

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        Graphics Display Assumes

                                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        nd

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Full Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        Full Rank Basis Matrix

                                                                                                                                                                                                                        All 0s in Bottom

                                                                                                                                                                                                                        ndX ddU ndS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        These Columns Get 0ed Out

                                                                                                                                                                                                                        ndX ddU nnS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        nnd 0

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                                        =ndX ndU nnS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Reduced Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        Also Some of These May be 0

                                                                                                                                                                                                                        ndX ndU nnS nn

                                                                                                                                                                                                                        tV

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                        0

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                        =

                                                                                                                                                                                                                        These Get 0ed Out

                                                                                                                                                                                                                        ndX rdU

                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                        0

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                        Find

                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                        Called Eigenvalues

                                                                                                                                                                                                                        Convenient Ordering

                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                        d

                                                                                                                                                                                                                        D

                                                                                                                                                                                                                        0

                                                                                                                                                                                                                        01

                                                                                                                                                                                                                        n 1

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                        d

                                                                                                                                                                                                                        D

                                                                                                                                                                                                                        0

                                                                                                                                                                                                                        01

                                                                                                                                                                                                                        ddB

                                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                                        So that ie

                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                        d

                                                                                                                                                                                                                        D

                                                                                                                                                                                                                        0

                                                                                                                                                                                                                        01

                                                                                                                                                                                                                        ddB

                                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                                        DBBX tBDBX

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                        (looks similar)

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                        Complex (uses )

                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                        X

                                                                                                                                                                                                                        BVU

                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                                                                        • SiZer Background
                                                                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          r

                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                          Useful Labeling

                                                                                                                                                                                                                          Singular Values in Increasing Order

                                                                                                                                                                                                                          Note singular values = 0 can be omitted

                                                                                                                                                                                                                          Let = of positive singular values

                                                                                                                                                                                                                          Then

                                                                                                                                                                                                                          Where are truncations of

                                                                                                                                                                                                                          trnrrrd VSUX

                                                                                                                                                                                                                          VSU

                                                                                                                                                                                                                          )min(1 dnss

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          Graphics Display Assumes

                                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          nd

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Full Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          Full Rank Basis Matrix

                                                                                                                                                                                                                          All 0s in Bottom

                                                                                                                                                                                                                          ndX ddU ndS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          These Columns Get 0ed Out

                                                                                                                                                                                                                          ndX ddU nnS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          nnd 0

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                                          =ndX ndU nnS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Reduced Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          Also Some of These May be 0

                                                                                                                                                                                                                          ndX ndU nnS nn

                                                                                                                                                                                                                          tV

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                                          0

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                          =

                                                                                                                                                                                                                          These Get 0ed Out

                                                                                                                                                                                                                          ndX rdU

                                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                                          0

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                          Find

                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                          Called Eigenvalues

                                                                                                                                                                                                                          Convenient Ordering

                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                          d

                                                                                                                                                                                                                          D

                                                                                                                                                                                                                          0

                                                                                                                                                                                                                          01

                                                                                                                                                                                                                          n 1

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                          d

                                                                                                                                                                                                                          D

                                                                                                                                                                                                                          0

                                                                                                                                                                                                                          01

                                                                                                                                                                                                                          ddB

                                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                                          So that ie

                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                          d

                                                                                                                                                                                                                          D

                                                                                                                                                                                                                          0

                                                                                                                                                                                                                          01

                                                                                                                                                                                                                          ddB

                                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                                          DBBX tBDBX

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                          (looks similar)

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                          Complex (uses )

                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                          X

                                                                                                                                                                                                                          BVU

                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                                                                          • SiZer Background
                                                                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            Graphics Display Assumes

                                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            nd

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Full Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            Full Rank Basis Matrix

                                                                                                                                                                                                                            All 0s in Bottom

                                                                                                                                                                                                                            ndX ddU ndS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            These Columns Get 0ed Out

                                                                                                                                                                                                                            ndX ddU nnS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            nnd 0

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                                            =ndX ndU nnS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Reduced Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            Also Some of These May be 0

                                                                                                                                                                                                                            ndX ndU nnS nn

                                                                                                                                                                                                                            tV

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                                            0

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                                            =

                                                                                                                                                                                                                            These Get 0ed Out

                                                                                                                                                                                                                            ndX rdU

                                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                                            0

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                            Find

                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                            Called Eigenvalues

                                                                                                                                                                                                                            Convenient Ordering

                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                            d

                                                                                                                                                                                                                            D

                                                                                                                                                                                                                            0

                                                                                                                                                                                                                            01

                                                                                                                                                                                                                            n 1

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                            d

                                                                                                                                                                                                                            D

                                                                                                                                                                                                                            0

                                                                                                                                                                                                                            01

                                                                                                                                                                                                                            ddB

                                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                                            So that ie

                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                            d

                                                                                                                                                                                                                            D

                                                                                                                                                                                                                            0

                                                                                                                                                                                                                            01

                                                                                                                                                                                                                            ddB

                                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                                            DBBX tBDBX

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                            (looks similar)

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                            Complex (uses )

                                                                                                                                                                                                                            VU

                                                                                                                                                                                                                            1i

                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                            VU

                                                                                                                                                                                                                            X

                                                                                                                                                                                                                            BVU

                                                                                                                                                                                                                            1i

                                                                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                                                                            • SiZer Background
                                                                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                                                              =

                                                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                                                              tV

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Full Representation

                                                                                                                                                                                                                              =

                                                                                                                                                                                                                              Full Rank Basis Matrix

                                                                                                                                                                                                                              All 0s in Bottom

                                                                                                                                                                                                                              ndX ddU ndS nn

                                                                                                                                                                                                                              tV

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                                              =

                                                                                                                                                                                                                              These Columns Get 0ed Out

                                                                                                                                                                                                                              ndX ddU nnS nn

                                                                                                                                                                                                                              tV

                                                                                                                                                                                                                              nnd 0

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                                              =ndX ndU nnS nn

                                                                                                                                                                                                                              tV

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Reduced Representation

                                                                                                                                                                                                                              =

                                                                                                                                                                                                                              Also Some of These May be 0

                                                                                                                                                                                                                              ndX ndU nnS nn

                                                                                                                                                                                                                              tV

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                                              0

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                                              =

                                                                                                                                                                                                                              These Get 0ed Out

                                                                                                                                                                                                                              ndX rdU

                                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                                              0

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              SVD Compact Representation

                                                                                                                                                                                                                              = ndX rdU

                                                                                                                                                                                                                              rrS nrtV

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                              Find

                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                              Called Eigenvalues

                                                                                                                                                                                                                              Convenient Ordering

                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                              d

                                                                                                                                                                                                                              D

                                                                                                                                                                                                                              0

                                                                                                                                                                                                                              01

                                                                                                                                                                                                                              n 1

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                              d

                                                                                                                                                                                                                              D

                                                                                                                                                                                                                              0

                                                                                                                                                                                                                              01

                                                                                                                                                                                                                              ddB

                                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                                              So that ie

                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                              d

                                                                                                                                                                                                                              D

                                                                                                                                                                                                                              0

                                                                                                                                                                                                                              01

                                                                                                                                                                                                                              ddB

                                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                                              DBBX tBDBX

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                              (looks similar)

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                              Complex (uses )

                                                                                                                                                                                                                              VU

                                                                                                                                                                                                                              1i

                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                              VU

                                                                                                                                                                                                                              X

                                                                                                                                                                                                                              BVU

                                                                                                                                                                                                                              1i

                                                                                                                                                                                                                              • Participant Presentations
                                                                                                                                                                                                                              • PCA to find clusters
                                                                                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                                                                                              • SiZer Background
                                                                                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                                                                                              • SiZer Overview
                                                                                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                                                                                              • New Subpopulation View
                                                                                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Full Representation

                                                                                                                                                                                                                                =

                                                                                                                                                                                                                                Full Rank Basis Matrix

                                                                                                                                                                                                                                All 0s in Bottom

                                                                                                                                                                                                                                ndX ddU ndS nn

                                                                                                                                                                                                                                tV

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                                =

                                                                                                                                                                                                                                These Columns Get 0ed Out

                                                                                                                                                                                                                                ndX ddU nnS nn

                                                                                                                                                                                                                                tV

                                                                                                                                                                                                                                nnd 0

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                                =ndX ndU nnS nn

                                                                                                                                                                                                                                tV

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Reduced Representation

                                                                                                                                                                                                                                =

                                                                                                                                                                                                                                Also Some of These May be 0

                                                                                                                                                                                                                                ndX ndU nnS nn

                                                                                                                                                                                                                                tV

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                                =

                                                                                                                                                                                                                                These Get 0ed Out

                                                                                                                                                                                                                                ndX rdU

                                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                SVD Compact Representation

                                                                                                                                                                                                                                = ndX rdU

                                                                                                                                                                                                                                rrS nrtV

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                Find

                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                                                                                Convenient Ordering

                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                n 1

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                                So that ie

                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                (looks similar)

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                Complex (uses )

                                                                                                                                                                                                                                VU

                                                                                                                                                                                                                                1i

                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                VU

                                                                                                                                                                                                                                X

                                                                                                                                                                                                                                BVU

                                                                                                                                                                                                                                1i

                                                                                                                                                                                                                                • Participant Presentations
                                                                                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                                                                                • SiZer Background
                                                                                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                                                                                • SiZer Overview
                                                                                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                                  =

                                                                                                                                                                                                                                  These Columns Get 0ed Out

                                                                                                                                                                                                                                  ndX ddU nnS nn

                                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                                  nnd 0

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                                  =ndX ndU nnS nn

                                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Reduced Representation

                                                                                                                                                                                                                                  =

                                                                                                                                                                                                                                  Also Some of These May be 0

                                                                                                                                                                                                                                  ndX ndU nnS nn

                                                                                                                                                                                                                                  tV

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                                  =

                                                                                                                                                                                                                                  These Get 0ed Out

                                                                                                                                                                                                                                  ndX rdU

                                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  SVD Compact Representation

                                                                                                                                                                                                                                  = ndX rdU

                                                                                                                                                                                                                                  rrS nrtV

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                  Find

                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                                  Called Eigenvalues

                                                                                                                                                                                                                                  Convenient Ordering

                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                  d

                                                                                                                                                                                                                                  D

                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                  01

                                                                                                                                                                                                                                  n 1

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                  d

                                                                                                                                                                                                                                  D

                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                  01

                                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                                  So that ie

                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                  d

                                                                                                                                                                                                                                  D

                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                  01

                                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                  (looks similar)

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                  Complex (uses )

                                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                                  X

                                                                                                                                                                                                                                  BVU

                                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                                                                                  • SiZer Background
                                                                                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                                                    =ndX ndU nnS nn

                                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    SVD Reduced Representation

                                                                                                                                                                                                                                    =

                                                                                                                                                                                                                                    Also Some of These May be 0

                                                                                                                                                                                                                                    ndX ndU nnS nn

                                                                                                                                                                                                                                    tV

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                                    =

                                                                                                                                                                                                                                    These Get 0ed Out

                                                                                                                                                                                                                                    ndX rdU

                                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    SVD Compact Representation

                                                                                                                                                                                                                                    = ndX rdU

                                                                                                                                                                                                                                    rrS nrtV

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                    Find

                                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                                    Called Eigenvalues

                                                                                                                                                                                                                                    Convenient Ordering

                                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                                    d

                                                                                                                                                                                                                                    D

                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                    01

                                                                                                                                                                                                                                    n 1

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                                    d

                                                                                                                                                                                                                                    D

                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                    01

                                                                                                                                                                                                                                    ddB

                                                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                                                    So that ie

                                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                                    d

                                                                                                                                                                                                                                    D

                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                    01

                                                                                                                                                                                                                                    ddB

                                                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                    (looks similar)

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                    Complex (uses )

                                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                                    X

                                                                                                                                                                                                                                    BVU

                                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                                                                                    • SiZer Background
                                                                                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      SVD Reduced Representation

                                                                                                                                                                                                                                      =

                                                                                                                                                                                                                                      Also Some of These May be 0

                                                                                                                                                                                                                                      ndX ndU nnS nn

                                                                                                                                                                                                                                      tV

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                                      0

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                                      =

                                                                                                                                                                                                                                      These Get 0ed Out

                                                                                                                                                                                                                                      ndX rdU

                                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                                      0

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      SVD Compact Representation

                                                                                                                                                                                                                                      = ndX rdU

                                                                                                                                                                                                                                      rrS nrtV

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                      Find

                                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                                      Called Eigenvalues

                                                                                                                                                                                                                                      Convenient Ordering

                                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                                      d

                                                                                                                                                                                                                                      D

                                                                                                                                                                                                                                      0

                                                                                                                                                                                                                                      01

                                                                                                                                                                                                                                      n 1

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                                      d

                                                                                                                                                                                                                                      D

                                                                                                                                                                                                                                      0

                                                                                                                                                                                                                                      01

                                                                                                                                                                                                                                      ddB

                                                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition

                                                                                                                                                                                                                                      For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                      Find a Diagonal Matrix

                                                                                                                                                                                                                                      And an Orthonormal Matrix

                                                                                                                                                                                                                                      (ie )

                                                                                                                                                                                                                                      So that ie

                                                                                                                                                                                                                                      ddX

                                                                                                                                                                                                                                      d

                                                                                                                                                                                                                                      D

                                                                                                                                                                                                                                      0

                                                                                                                                                                                                                                      01

                                                                                                                                                                                                                                      ddB

                                                                                                                                                                                                                                      ddtt IBBBB

                                                                                                                                                                                                                                      DBBX tBDBX

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                      (looks similar)

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                      Complex (uses )

                                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                                      X

                                                                                                                                                                                                                                      BVU

                                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                                                                                      • SiZer Background
                                                                                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                                        0

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                                        =

                                                                                                                                                                                                                                        These Get 0ed Out

                                                                                                                                                                                                                                        ndX rdU

                                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                                        0

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        SVD Compact Representation

                                                                                                                                                                                                                                        = ndX rdU

                                                                                                                                                                                                                                        rrS nrtV

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                        Find

                                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                                        Called Eigenvalues

                                                                                                                                                                                                                                        Convenient Ordering

                                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                                        d

                                                                                                                                                                                                                                        D

                                                                                                                                                                                                                                        0

                                                                                                                                                                                                                                        01

                                                                                                                                                                                                                                        n 1

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                                        d

                                                                                                                                                                                                                                        D

                                                                                                                                                                                                                                        0

                                                                                                                                                                                                                                        01

                                                                                                                                                                                                                                        ddB

                                                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition

                                                                                                                                                                                                                                        For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                        Find a Diagonal Matrix

                                                                                                                                                                                                                                        And an Orthonormal Matrix

                                                                                                                                                                                                                                        (ie )

                                                                                                                                                                                                                                        So that ie

                                                                                                                                                                                                                                        ddX

                                                                                                                                                                                                                                        d

                                                                                                                                                                                                                                        D

                                                                                                                                                                                                                                        0

                                                                                                                                                                                                                                        01

                                                                                                                                                                                                                                        ddB

                                                                                                                                                                                                                                        ddtt IBBBB

                                                                                                                                                                                                                                        DBBX tBDBX

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                        (looks similar)

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                        Complex (uses )

                                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                                        X

                                                                                                                                                                                                                                        BVU

                                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                                                                                        • SiZer Background
                                                                                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                                          =

                                                                                                                                                                                                                                          These Get 0ed Out

                                                                                                                                                                                                                                          ndX rdU

                                                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                                                          0

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          SVD Compact Representation

                                                                                                                                                                                                                                          = ndX rdU

                                                                                                                                                                                                                                          rrS nrtV

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                          Find

                                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                                          Called Eigenvalues

                                                                                                                                                                                                                                          Convenient Ordering

                                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                                          d

                                                                                                                                                                                                                                          D

                                                                                                                                                                                                                                          0

                                                                                                                                                                                                                                          01

                                                                                                                                                                                                                                          n 1

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                                          d

                                                                                                                                                                                                                                          D

                                                                                                                                                                                                                                          0

                                                                                                                                                                                                                                          01

                                                                                                                                                                                                                                          ddB

                                                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition

                                                                                                                                                                                                                                          For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                          Find a Diagonal Matrix

                                                                                                                                                                                                                                          And an Orthonormal Matrix

                                                                                                                                                                                                                                          (ie )

                                                                                                                                                                                                                                          So that ie

                                                                                                                                                                                                                                          ddX

                                                                                                                                                                                                                                          d

                                                                                                                                                                                                                                          D

                                                                                                                                                                                                                                          0

                                                                                                                                                                                                                                          01

                                                                                                                                                                                                                                          ddB

                                                                                                                                                                                                                                          ddtt IBBBB

                                                                                                                                                                                                                                          DBBX tBDBX

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                          (looks similar)

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                          Complex (uses )

                                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                                          X

                                                                                                                                                                                                                                          BVU

                                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                                                                                          • SiZer Background
                                                                                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            SVD Compact Representation

                                                                                                                                                                                                                                            = ndX rdU

                                                                                                                                                                                                                                            rrS nrtV

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                            Find

                                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                                            Called Eigenvalues

                                                                                                                                                                                                                                            Convenient Ordering

                                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                                            d

                                                                                                                                                                                                                                            D

                                                                                                                                                                                                                                            0

                                                                                                                                                                                                                                            01

                                                                                                                                                                                                                                            n 1

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                                            d

                                                                                                                                                                                                                                            D

                                                                                                                                                                                                                                            0

                                                                                                                                                                                                                                            01

                                                                                                                                                                                                                                            ddB

                                                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition

                                                                                                                                                                                                                                            For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                            Find a Diagonal Matrix

                                                                                                                                                                                                                                            And an Orthonormal Matrix

                                                                                                                                                                                                                                            (ie )

                                                                                                                                                                                                                                            So that ie

                                                                                                                                                                                                                                            ddX

                                                                                                                                                                                                                                            d

                                                                                                                                                                                                                                            D

                                                                                                                                                                                                                                            0

                                                                                                                                                                                                                                            01

                                                                                                                                                                                                                                            ddB

                                                                                                                                                                                                                                            ddtt IBBBB

                                                                                                                                                                                                                                            DBBX tBDBX

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                            (looks similar)

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                            Complex (uses )

                                                                                                                                                                                                                                            VU

                                                                                                                                                                                                                                            1i

                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                            VU

                                                                                                                                                                                                                                            X

                                                                                                                                                                                                                                            BVU

                                                                                                                                                                                                                                            1i

                                                                                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                                                                                            • SiZer Background
                                                                                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                              Find

                                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                                              Called Eigenvalues

                                                                                                                                                                                                                                              Convenient Ordering

                                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                                              d

                                                                                                                                                                                                                                              D

                                                                                                                                                                                                                                              0

                                                                                                                                                                                                                                              01

                                                                                                                                                                                                                                              n 1

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                                              d

                                                                                                                                                                                                                                              D

                                                                                                                                                                                                                                              0

                                                                                                                                                                                                                                              01

                                                                                                                                                                                                                                              ddB

                                                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition

                                                                                                                                                                                                                                              For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                              Find a Diagonal Matrix

                                                                                                                                                                                                                                              And an Orthonormal Matrix

                                                                                                                                                                                                                                              (ie )

                                                                                                                                                                                                                                              So that ie

                                                                                                                                                                                                                                              ddX

                                                                                                                                                                                                                                              d

                                                                                                                                                                                                                                              D

                                                                                                                                                                                                                                              0

                                                                                                                                                                                                                                              01

                                                                                                                                                                                                                                              ddB

                                                                                                                                                                                                                                              ddtt IBBBB

                                                                                                                                                                                                                                              DBBX tBDBX

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                              (looks similar)

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                              Complex (uses )

                                                                                                                                                                                                                                              VU

                                                                                                                                                                                                                                              1i

                                                                                                                                                                                                                                              Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                              Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                              (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                              Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                              VU

                                                                                                                                                                                                                                              X

                                                                                                                                                                                                                                              BVU

                                                                                                                                                                                                                                              1i

                                                                                                                                                                                                                                              • Participant Presentations
                                                                                                                                                                                                                                              • PCA to find clusters
                                                                                                                                                                                                                                              • Statistical Smoothing
                                                                                                                                                                                                                                              • Kernel Density Estimation
                                                                                                                                                                                                                                              • Scatterplot Smoothing
                                                                                                                                                                                                                                              • Statistical Smoothing (2)
                                                                                                                                                                                                                                              • SiZer Background
                                                                                                                                                                                                                                              • SiZer Background (2)
                                                                                                                                                                                                                                              • SiZer Background (3)
                                                                                                                                                                                                                                              • SiZer Background (4)
                                                                                                                                                                                                                                              • SiZer Background (5)
                                                                                                                                                                                                                                              • SiZer Overview
                                                                                                                                                                                                                                              • PCA to find clusters (2)
                                                                                                                                                                                                                                              • PCA to find clusters (3)
                                                                                                                                                                                                                                              • PCA to find clusters (4)
                                                                                                                                                                                                                                              • PCA to find clusters (5)
                                                                                                                                                                                                                                              • PCA to find clusters (6)
                                                                                                                                                                                                                                              • PCA to find clusters (7)
                                                                                                                                                                                                                                              • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                              • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                              • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                              • Frequency 2 Analysis
                                                                                                                                                                                                                                              • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                              • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                              • Yeast Cell Cycle
                                                                                                                                                                                                                                              • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                              • Reclassification of Major Genes
                                                                                                                                                                                                                                              • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                              • New Subpopulation View
                                                                                                                                                                                                                                              • New Subpopulation View (2)
                                                                                                                                                                                                                                              • Detailed Look at PCA
                                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                              • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                              • An Interesting Historical Note
                                                                                                                                                                                                                                              • An Interesting Historical Note (2)
                                                                                                                                                                                                                                              • Detailed Look at PCA (2)
                                                                                                                                                                                                                                              • Detailed Look at PCA (3)
                                                                                                                                                                                                                                              • Review of Linear Algebra
                                                                                                                                                                                                                                              • Review of Linear Algebra (2)
                                                                                                                                                                                                                                              • Review of Linear Algebra (3)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                              • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                                Called Eigenvalues

                                                                                                                                                                                                                                                Convenient Ordering

                                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                                n 1

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition

                                                                                                                                                                                                                                                For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                Find a Diagonal Matrix

                                                                                                                                                                                                                                                And an Orthonormal Matrix

                                                                                                                                                                                                                                                (ie )

                                                                                                                                                                                                                                                So that ie

                                                                                                                                                                                                                                                ddX

                                                                                                                                                                                                                                                d

                                                                                                                                                                                                                                                D

                                                                                                                                                                                                                                                0

                                                                                                                                                                                                                                                01

                                                                                                                                                                                                                                                ddB

                                                                                                                                                                                                                                                ddtt IBBBB

                                                                                                                                                                                                                                                DBBX tBDBX

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                (looks similar)

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                Complex (uses )

                                                                                                                                                                                                                                                VU

                                                                                                                                                                                                                                                1i

                                                                                                                                                                                                                                                Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                VU

                                                                                                                                                                                                                                                X

                                                                                                                                                                                                                                                BVU

                                                                                                                                                                                                                                                1i

                                                                                                                                                                                                                                                • Participant Presentations
                                                                                                                                                                                                                                                • PCA to find clusters
                                                                                                                                                                                                                                                • Statistical Smoothing
                                                                                                                                                                                                                                                • Kernel Density Estimation
                                                                                                                                                                                                                                                • Scatterplot Smoothing
                                                                                                                                                                                                                                                • Statistical Smoothing (2)
                                                                                                                                                                                                                                                • SiZer Background
                                                                                                                                                                                                                                                • SiZer Background (2)
                                                                                                                                                                                                                                                • SiZer Background (3)
                                                                                                                                                                                                                                                • SiZer Background (4)
                                                                                                                                                                                                                                                • SiZer Background (5)
                                                                                                                                                                                                                                                • SiZer Overview
                                                                                                                                                                                                                                                • PCA to find clusters (2)
                                                                                                                                                                                                                                                • PCA to find clusters (3)
                                                                                                                                                                                                                                                • PCA to find clusters (4)
                                                                                                                                                                                                                                                • PCA to find clusters (5)
                                                                                                                                                                                                                                                • PCA to find clusters (6)
                                                                                                                                                                                                                                                • PCA to find clusters (7)
                                                                                                                                                                                                                                                • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                • Frequency 2 Analysis
                                                                                                                                                                                                                                                • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                • Yeast Cell Cycle
                                                                                                                                                                                                                                                • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                • Reclassification of Major Genes
                                                                                                                                                                                                                                                • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                • New Subpopulation View
                                                                                                                                                                                                                                                • New Subpopulation View (2)
                                                                                                                                                                                                                                                • Detailed Look at PCA
                                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                • An Interesting Historical Note
                                                                                                                                                                                                                                                • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                • Review of Linear Algebra
                                                                                                                                                                                                                                                • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                                  d

                                                                                                                                                                                                                                                  D

                                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                                  01

                                                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition

                                                                                                                                                                                                                                                  For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                  Find a Diagonal Matrix

                                                                                                                                                                                                                                                  And an Orthonormal Matrix

                                                                                                                                                                                                                                                  (ie )

                                                                                                                                                                                                                                                  So that ie

                                                                                                                                                                                                                                                  ddX

                                                                                                                                                                                                                                                  d

                                                                                                                                                                                                                                                  D

                                                                                                                                                                                                                                                  0

                                                                                                                                                                                                                                                  01

                                                                                                                                                                                                                                                  ddB

                                                                                                                                                                                                                                                  ddtt IBBBB

                                                                                                                                                                                                                                                  DBBX tBDBX

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                  (looks similar)

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                  Complex (uses )

                                                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                                                  Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                  Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                  (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                  Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                  VU

                                                                                                                                                                                                                                                  X

                                                                                                                                                                                                                                                  BVU

                                                                                                                                                                                                                                                  1i

                                                                                                                                                                                                                                                  • Participant Presentations
                                                                                                                                                                                                                                                  • PCA to find clusters
                                                                                                                                                                                                                                                  • Statistical Smoothing
                                                                                                                                                                                                                                                  • Kernel Density Estimation
                                                                                                                                                                                                                                                  • Scatterplot Smoothing
                                                                                                                                                                                                                                                  • Statistical Smoothing (2)
                                                                                                                                                                                                                                                  • SiZer Background
                                                                                                                                                                                                                                                  • SiZer Background (2)
                                                                                                                                                                                                                                                  • SiZer Background (3)
                                                                                                                                                                                                                                                  • SiZer Background (4)
                                                                                                                                                                                                                                                  • SiZer Background (5)
                                                                                                                                                                                                                                                  • SiZer Overview
                                                                                                                                                                                                                                                  • PCA to find clusters (2)
                                                                                                                                                                                                                                                  • PCA to find clusters (3)
                                                                                                                                                                                                                                                  • PCA to find clusters (4)
                                                                                                                                                                                                                                                  • PCA to find clusters (5)
                                                                                                                                                                                                                                                  • PCA to find clusters (6)
                                                                                                                                                                                                                                                  • PCA to find clusters (7)
                                                                                                                                                                                                                                                  • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                  • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                  • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                  • Frequency 2 Analysis
                                                                                                                                                                                                                                                  • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                  • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                  • Yeast Cell Cycle
                                                                                                                                                                                                                                                  • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                  • Reclassification of Major Genes
                                                                                                                                                                                                                                                  • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                  • New Subpopulation View
                                                                                                                                                                                                                                                  • New Subpopulation View (2)
                                                                                                                                                                                                                                                  • Detailed Look at PCA
                                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                  • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                  • An Interesting Historical Note
                                                                                                                                                                                                                                                  • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                  • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                  • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                  • Review of Linear Algebra
                                                                                                                                                                                                                                                  • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                  • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                    Eigenvalue Decomposition

                                                                                                                                                                                                                                                    For a (Symmetric) Square Matrix

                                                                                                                                                                                                                                                    Find a Diagonal Matrix

                                                                                                                                                                                                                                                    And an Orthonormal Matrix

                                                                                                                                                                                                                                                    (ie )

                                                                                                                                                                                                                                                    So that ie

                                                                                                                                                                                                                                                    ddX

                                                                                                                                                                                                                                                    d

                                                                                                                                                                                                                                                    D

                                                                                                                                                                                                                                                    0

                                                                                                                                                                                                                                                    01

                                                                                                                                                                                                                                                    ddB

                                                                                                                                                                                                                                                    ddtt IBBBB

                                                                                                                                                                                                                                                    DBBX tBDBX

                                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                    (looks similar)

                                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                    Complex (uses )

                                                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                                                    Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                    Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                    (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                    Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                    VU

                                                                                                                                                                                                                                                    X

                                                                                                                                                                                                                                                    BVU

                                                                                                                                                                                                                                                    1i

                                                                                                                                                                                                                                                    • Participant Presentations
                                                                                                                                                                                                                                                    • PCA to find clusters
                                                                                                                                                                                                                                                    • Statistical Smoothing
                                                                                                                                                                                                                                                    • Kernel Density Estimation
                                                                                                                                                                                                                                                    • Scatterplot Smoothing
                                                                                                                                                                                                                                                    • Statistical Smoothing (2)
                                                                                                                                                                                                                                                    • SiZer Background
                                                                                                                                                                                                                                                    • SiZer Background (2)
                                                                                                                                                                                                                                                    • SiZer Background (3)
                                                                                                                                                                                                                                                    • SiZer Background (4)
                                                                                                                                                                                                                                                    • SiZer Background (5)
                                                                                                                                                                                                                                                    • SiZer Overview
                                                                                                                                                                                                                                                    • PCA to find clusters (2)
                                                                                                                                                                                                                                                    • PCA to find clusters (3)
                                                                                                                                                                                                                                                    • PCA to find clusters (4)
                                                                                                                                                                                                                                                    • PCA to find clusters (5)
                                                                                                                                                                                                                                                    • PCA to find clusters (6)
                                                                                                                                                                                                                                                    • PCA to find clusters (7)
                                                                                                                                                                                                                                                    • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                    • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                    • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                    • Frequency 2 Analysis
                                                                                                                                                                                                                                                    • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                    • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                    • Yeast Cell Cycle
                                                                                                                                                                                                                                                    • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                    • Reclassification of Major Genes
                                                                                                                                                                                                                                                    • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                    • New Subpopulation View
                                                                                                                                                                                                                                                    • New Subpopulation View (2)
                                                                                                                                                                                                                                                    • Detailed Look at PCA
                                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                    • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                    • An Interesting Historical Note
                                                                                                                                                                                                                                                    • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                    • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                    • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                    • Review of Linear Algebra
                                                                                                                                                                                                                                                    • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                    • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                      (looks similar)

                                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                      Complex (uses )

                                                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                                                      Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                      Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                      (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                      Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                      VU

                                                                                                                                                                                                                                                      X

                                                                                                                                                                                                                                                      BVU

                                                                                                                                                                                                                                                      1i

                                                                                                                                                                                                                                                      • Participant Presentations
                                                                                                                                                                                                                                                      • PCA to find clusters
                                                                                                                                                                                                                                                      • Statistical Smoothing
                                                                                                                                                                                                                                                      • Kernel Density Estimation
                                                                                                                                                                                                                                                      • Scatterplot Smoothing
                                                                                                                                                                                                                                                      • Statistical Smoothing (2)
                                                                                                                                                                                                                                                      • SiZer Background
                                                                                                                                                                                                                                                      • SiZer Background (2)
                                                                                                                                                                                                                                                      • SiZer Background (3)
                                                                                                                                                                                                                                                      • SiZer Background (4)
                                                                                                                                                                                                                                                      • SiZer Background (5)
                                                                                                                                                                                                                                                      • SiZer Overview
                                                                                                                                                                                                                                                      • PCA to find clusters (2)
                                                                                                                                                                                                                                                      • PCA to find clusters (3)
                                                                                                                                                                                                                                                      • PCA to find clusters (4)
                                                                                                                                                                                                                                                      • PCA to find clusters (5)
                                                                                                                                                                                                                                                      • PCA to find clusters (6)
                                                                                                                                                                                                                                                      • PCA to find clusters (7)
                                                                                                                                                                                                                                                      • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                      • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                      • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                      • Frequency 2 Analysis
                                                                                                                                                                                                                                                      • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                      • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                      • Yeast Cell Cycle
                                                                                                                                                                                                                                                      • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                      • Reclassification of Major Genes
                                                                                                                                                                                                                                                      • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                      • New Subpopulation View
                                                                                                                                                                                                                                                      • New Subpopulation View (2)
                                                                                                                                                                                                                                                      • Detailed Look at PCA
                                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                      • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                      • An Interesting Historical Note
                                                                                                                                                                                                                                                      • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                      • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                      • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                      • Review of Linear Algebra
                                                                                                                                                                                                                                                      • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                      • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs VU

                                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                        Complex (uses )

                                                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                                                        Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                        Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                        (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                        Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                        VU

                                                                                                                                                                                                                                                        X

                                                                                                                                                                                                                                                        BVU

                                                                                                                                                                                                                                                        1i

                                                                                                                                                                                                                                                        • Participant Presentations
                                                                                                                                                                                                                                                        • PCA to find clusters
                                                                                                                                                                                                                                                        • Statistical Smoothing
                                                                                                                                                                                                                                                        • Kernel Density Estimation
                                                                                                                                                                                                                                                        • Scatterplot Smoothing
                                                                                                                                                                                                                                                        • Statistical Smoothing (2)
                                                                                                                                                                                                                                                        • SiZer Background
                                                                                                                                                                                                                                                        • SiZer Background (2)
                                                                                                                                                                                                                                                        • SiZer Background (3)
                                                                                                                                                                                                                                                        • SiZer Background (4)
                                                                                                                                                                                                                                                        • SiZer Background (5)
                                                                                                                                                                                                                                                        • SiZer Overview
                                                                                                                                                                                                                                                        • PCA to find clusters (2)
                                                                                                                                                                                                                                                        • PCA to find clusters (3)
                                                                                                                                                                                                                                                        • PCA to find clusters (4)
                                                                                                                                                                                                                                                        • PCA to find clusters (5)
                                                                                                                                                                                                                                                        • PCA to find clusters (6)
                                                                                                                                                                                                                                                        • PCA to find clusters (7)
                                                                                                                                                                                                                                                        • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                        • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                        • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                        • Frequency 2 Analysis
                                                                                                                                                                                                                                                        • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                        • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                        • Yeast Cell Cycle
                                                                                                                                                                                                                                                        • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                        • Reclassification of Major Genes
                                                                                                                                                                                                                                                        • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                        • New Subpopulation View
                                                                                                                                                                                                                                                        • New Subpopulation View (2)
                                                                                                                                                                                                                                                        • Detailed Look at PCA
                                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                        • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                        • An Interesting Historical Note
                                                                                                                                                                                                                                                        • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                        • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                        • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                        • Review of Linear Algebra
                                                                                                                                                                                                                                                        • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                        • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                          Complex (uses )

                                                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                                                          Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                          Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                          (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                          Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                          VU

                                                                                                                                                                                                                                                          X

                                                                                                                                                                                                                                                          BVU

                                                                                                                                                                                                                                                          1i

                                                                                                                                                                                                                                                          • Participant Presentations
                                                                                                                                                                                                                                                          • PCA to find clusters
                                                                                                                                                                                                                                                          • Statistical Smoothing
                                                                                                                                                                                                                                                          • Kernel Density Estimation
                                                                                                                                                                                                                                                          • Scatterplot Smoothing
                                                                                                                                                                                                                                                          • Statistical Smoothing (2)
                                                                                                                                                                                                                                                          • SiZer Background
                                                                                                                                                                                                                                                          • SiZer Background (2)
                                                                                                                                                                                                                                                          • SiZer Background (3)
                                                                                                                                                                                                                                                          • SiZer Background (4)
                                                                                                                                                                                                                                                          • SiZer Background (5)
                                                                                                                                                                                                                                                          • SiZer Overview
                                                                                                                                                                                                                                                          • PCA to find clusters (2)
                                                                                                                                                                                                                                                          • PCA to find clusters (3)
                                                                                                                                                                                                                                                          • PCA to find clusters (4)
                                                                                                                                                                                                                                                          • PCA to find clusters (5)
                                                                                                                                                                                                                                                          • PCA to find clusters (6)
                                                                                                                                                                                                                                                          • PCA to find clusters (7)
                                                                                                                                                                                                                                                          • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                          • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                          • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                          • Frequency 2 Analysis
                                                                                                                                                                                                                                                          • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                          • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                          • Yeast Cell Cycle
                                                                                                                                                                                                                                                          • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                          • Reclassification of Major Genes
                                                                                                                                                                                                                                                          • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                          • New Subpopulation View
                                                                                                                                                                                                                                                          • New Subpopulation View (2)
                                                                                                                                                                                                                                                          • Detailed Look at PCA
                                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                          • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                          • An Interesting Historical Note
                                                                                                                                                                                                                                                          • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                          • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                          • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                          • Review of Linear Algebra
                                                                                                                                                                                                                                                          • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                          • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                            Review of Linear Algebra (Cont)

                                                                                                                                                                                                                                                            Eigenvalue Decomposition (cont)bull Relation to Singular Value Decomposition

                                                                                                                                                                                                                                                            (looks similar)bull Eigenvalue Decomposition ldquoLooks Harderrdquobull Since Needs bull Price is Eigenvalue Decomprsquon is Generally

                                                                                                                                                                                                                                                            Complex (uses )bull Except for Square and Symmetricbull Then Eigenvalue Decomp is Real Valuedbull Thus is the Singrsquor Value Decomp with

                                                                                                                                                                                                                                                            VU

                                                                                                                                                                                                                                                            X

                                                                                                                                                                                                                                                            BVU

                                                                                                                                                                                                                                                            1i

                                                                                                                                                                                                                                                            • Participant Presentations
                                                                                                                                                                                                                                                            • PCA to find clusters
                                                                                                                                                                                                                                                            • Statistical Smoothing
                                                                                                                                                                                                                                                            • Kernel Density Estimation
                                                                                                                                                                                                                                                            • Scatterplot Smoothing
                                                                                                                                                                                                                                                            • Statistical Smoothing (2)
                                                                                                                                                                                                                                                            • SiZer Background
                                                                                                                                                                                                                                                            • SiZer Background (2)
                                                                                                                                                                                                                                                            • SiZer Background (3)
                                                                                                                                                                                                                                                            • SiZer Background (4)
                                                                                                                                                                                                                                                            • SiZer Background (5)
                                                                                                                                                                                                                                                            • SiZer Overview
                                                                                                                                                                                                                                                            • PCA to find clusters (2)
                                                                                                                                                                                                                                                            • PCA to find clusters (3)
                                                                                                                                                                                                                                                            • PCA to find clusters (4)
                                                                                                                                                                                                                                                            • PCA to find clusters (5)
                                                                                                                                                                                                                                                            • PCA to find clusters (6)
                                                                                                                                                                                                                                                            • PCA to find clusters (7)
                                                                                                                                                                                                                                                            • Recall Yeast Cell Cycle Data
                                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View
                                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (2)
                                                                                                                                                                                                                                                            • Yeast Cell Cycle Data FDA View (3)
                                                                                                                                                                                                                                                            • Yeast Cell Cycles Freq 2 Proj
                                                                                                                                                                                                                                                            • Frequency 2 Analysis
                                                                                                                                                                                                                                                            • Frequency 2 Analysis (2)
                                                                                                                                                                                                                                                            • Frequency 2 Analysis (3)
                                                                                                                                                                                                                                                            • Yeast Cell Cycle
                                                                                                                                                                                                                                                            • SiZer Study of Distrsquon of Angles
                                                                                                                                                                                                                                                            • Reclassification of Major Genes
                                                                                                                                                                                                                                                            • Compare to Previous Classifrsquon
                                                                                                                                                                                                                                                            • New Subpopulation View
                                                                                                                                                                                                                                                            • New Subpopulation View (2)
                                                                                                                                                                                                                                                            • Detailed Look at PCA
                                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming
                                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (2)
                                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (3)
                                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (4)
                                                                                                                                                                                                                                                            • PCA Rediscovery ndash Renaming (5)
                                                                                                                                                                                                                                                            • An Interesting Historical Note
                                                                                                                                                                                                                                                            • An Interesting Historical Note (2)
                                                                                                                                                                                                                                                            • Detailed Look at PCA (2)
                                                                                                                                                                                                                                                            • Detailed Look at PCA (3)
                                                                                                                                                                                                                                                            • Review of Linear Algebra
                                                                                                                                                                                                                                                            • Review of Linear Algebra (2)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (3)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (2)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (3)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (4)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (5)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (6)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (7)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (8)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (9)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (10)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (11)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (12)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (13)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (14)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (15)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (16)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (17)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (18)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (19)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (20)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (21)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (22)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (23)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (24)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (25)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (26)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (27)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (28)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (29)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (30)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (31)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (32)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (33)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (34)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (35)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (36)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (37)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (38)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (39)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (40)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (41)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (42)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (43)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (44)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (45)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (46)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (47)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (48)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (49)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (50)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (51)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (52)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (53)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (54)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (55)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (56)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (57)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (58)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (59)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (60)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (61)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (62)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (63)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (64)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (65)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (66)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (67)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (68)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (69)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (70)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (71)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (72)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (73)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (74)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (75)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (76)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (77)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (78)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (79)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (80)
                                                                                                                                                                                                                                                            • Review of Linear Algebra (Cont) (81)

                                                                                                                                                                                                                                                              top related

                                                                                                                                                                                                                                                              BIRMINGHAM EDUCATION ASSOCIATION TENTATIVE AGREEMENT FOR...
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Final Tentative Agreement
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative Proposal
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative teachers contract
                                                                                                                                                                                                                                                              Education
                                                                                                                                                                                                                                                              Tentative Product
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative Seniority List of Junior Engineer...
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Appendix D: Geotechnical Investigation, Tentative Tract...
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              ENROLLED ORIGINAL
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative Agenda ukuran Amplop - Universitas...
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Woods Tentative
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Course Outline (Tentative)
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Presentation Schedule Tentative)
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative Conference Programme World Press Freedom … ·.....
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Full page photo Training under Project... · karakoti ajay....
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Korba Tentative
                                                                                                                                                                                                                                                              Documents
                                                                                                                                                                                                                                                              Tentative Final Monograph
                                                                                                                                                                                                                                                              Documents